Haystack

GSoC Contributor Guidance

Mon, 01 Jan 0001 00:00:00 +0000

What is Haystack

Haystack is the open source Python framework by deepset for building custom apps with large language models (LLMs). It lets you quickly try out the latest models in natural language processing (NLP) while being flexible and easy to use. Our inspiring community of users and builders has helped shape Haystack into what it is today: a complete framework for building production-ready NLP apps.

For more details about Haystack:

GSoC Project Ideas

Mon, 01 Jan 0001 00:00:00 +0000

spaCy Integration in Haystack: Seamless NLP Pipelines

Proposed mentors: Madeesh Kannan, Stefano Fiorucci
Languages/skills: Python, spaCy
Estimated Project Length: 175 hours
Difficulty: medium

spaCy and Haystack are two NLP frameworks with different strengths that complement each other but currently they can hardly be used together. The goal of this project is a spaCy integration in Haystack, providing NLP practitioners, developers, and researchers with the flexibility to harness the combined power of both frameworks seamlessly in their NLP workflows. The implementation of this integration will allow users to easily incorporate spaCy components, such as tokenization, feature extraction, named entity recognition (NER), and part-of-speech (POS) tagging, to enhance the preprocessing capabilities of Haystack. Project participants could also focus in particular on efficient processing of large-scale text data, taking advantage of spaCy’s parallel processing capabilities for speed and scalability.

What is Haystack?

Mon, 01 Jan 0001 00:00:00 +0000

Haystack is an open-source AI orchestration framework built by deepset, enabling Python developers to build production-ready AI agents, multimodal applications, and advanced RAG systems.

Built for scalable context engineering, Haystack gives you control over how information moves through your system, from retrieval and tool use to memory and model execution. Haystack structures agents and applications as explicit, modular pipelines composed of retrievers, routers, memory layers, tools, evaluators, and generators. This modular architecture allows each component to be tested, replaced, and improved independently. As a result, you can ship faster and continuously evolve applications in production.

Collaborate with the Haystack Core Team

Mon, 01 Jan 0001 00:00:00 +0000

Get Started

Mon, 01 Jan 0001 00:00:00 +0000

Haystack is an open-source AI framework to build custom production-grade LLM applications such as AI agents, powerful RAG applications, and scalable search systems.

Installation

Use pip to install Haystack:

pip install haystack-ai

For more details, refer to our documentation.

Docs: Installation

Prerequisites

To run the example, you’ll need:

Demos

Mon, 01 Jan 0001 00:00:00 +0000

Check out demos built with Haystack!

AutoQuizzer

Try out our AutoQuizzer demo built with Haystack and Llama 3 8B Instruct! Generate a quiz from a URL, solve the quiz, or let the LLM play it.

🤗 AutoQuizzer HF Space

Other Demos

Here are some demos built with Haystack for various use cases:

Support for Talks, Workshops, and Meetups

Mon, 01 Jan 0001 00:00:00 +0000

Roadmap

Mon, 01 Jan 0001 00:00:00 +0000

We believe open-source is more than open-source code. It’s about the collaboration, transparency, and trust. Therefore, we decided to be as open as possible with our roadmap. In fact, you can see all of our quarterly roadmap items on GitHub. We hope this helps to clarify the direction of the open-source project and inspires discussions in the community.

Our most recent roadmap is hosted on Github. Here you will find the high-level projects that we have planned for the upcoming quarters. We update it regularly and refine the projects as their start date approaches.

Travel Support

Mon, 01 Jan 0001 00:00:00 +0000

Creating Your First QA Pipeline with Retrieval-Augmentation

Wed, 17 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 10 minutes
Components Used: InMemoryDocumentStore, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, InMemoryEmbeddingRetriever, ChatPromptBuilder, and a ChatGenerator such as OpenAIChatGenerator, MistralChatGenerator, or TransformersChatGenerator.
Prerequisites: Access to a large language model, either an API key from a provider or a locally or on-premises hosted model (for example on Colab runtime).
Goal: After completing this tutorial, you’ll have learned the new prompt syntax and how to use ChatPromptBuilder with a ChatGenerator to build a generative question-answering pipeline with retrieval-augmentation.

Overview

This tutorial shows you how to create a generative question-answering pipeline using the retrieval-augmentation ( RAG) approach with Haystack. The process involves four main components: SentenceTransformersTextEmbedder for creating an embedding for the user query, InMemoryEmbeddingRetriever for fetching relevant documents, ChatPromptBuilder for creating a template prompt, and a ChatGenerator for generating the final answer.

Amplify Your Work

Mon, 01 Jan 0001 00:00:00 +0000

Use Cases

Mon, 01 Jan 0001 00:00:00 +0000

Semantic Search System

Take the leap from using keyword search on your own documents to semantic search with Haystack.

Store your documents in the database of your choice (Elasticsearch, SQL, in memory, FAISS)
Perform question driven queries.

Expect to see results that highlight the very sentence that contains the answer to your question. Thanks to the power of Transformer based language models, results are chosen based on compatibility in meaning rather than lexical overlap.

Filtering Documents with Metadata

Wed, 03 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 5 minutes
Components Used: InMemoryDocumentStore, InMemoryBM25Retriever
Prerequisites: None
Goal: Filter documents in a document store based on given metadata

Overview

📚 Useful Documentation: Metadata Filtering

Although new retrieval techniques are great, sometimes you just know that you want to perform search on a specific group of documents in your document store. This can be anything from all the documents that are related to a specific user, or that were published after a certain date and so on. Metadata filtering is very useful in these situations. In this tutorial, we will create a few simple documents containing information about Haystack, where the metadata includes information on what version of Haystack the information relates to. We will then do metadata filtering to make sure we are answering the question based only on information about Haystack 2.0.

Preprocessing Different File Types

Wed, 17 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 15 minutes
Goal: After completing this tutorial, you’ll have learned how to build an indexing pipeline that will preprocess files based on their file type, using the FileTypeRouter.

💡 (Optional): After creating the indexing pipeline in this tutorial, there is an optional section that shows you how to create a RAG pipeline on top of the document store you just created. You must have a Hugging Face API Key for this section

Build a Tool-Calling Agent

Fri, 12 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 15 minutes
Components Used: Agent, OpenAIChatGenerator, SerperDevWebSearch, ComponentTool, PipelineTool
Prerequisites: You must have an OpenAI API Key and a SerperDev API Key
Goal: After completing this tutorial, you’ll have learned how to create an Agent that can use both components and pipelines as tools to answer questions and perform tasks.

Overview

In this tutorial, you’ll learn how to create an agent that can use tools to answer questions and perform tasks. We’ll explore two approaches:

Creating Custom SuperComponents

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 20 minutes
Concepts and Components Used: @super_component, Pipeline, DocumentJoiner, SentenceTransformersTextEmbedder, InMemoryBM25Retriever, InMemoryEmbeddingRetriever, SentenceTransformersSimilarityRanker
Goal: After completing this tutorial, you’ll have learned how to create custom SuperComponents using the @super_component decorator to simplify complex pipelines and make them reusable as components.

Overview

In this tutorial, you’ll learn how to create custom SuperComponents using the @super_component decorator. SuperComponents are a powerful way to encapsulate complex pipelines into reusable components with simplified interfaces.

We’ll explore several examples:

Embedding Metadata for Improved Retrieval

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 10 minutes
Components Used: InMemoryDocumentStore, InMemoryEmbeddingRetriever, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder
Goal: After completing this tutorial, you’ll have learned how to embed metadata information while indexing documents, to improve retrieval.

⚠️ Note of caution: The method showcased in this tutorial is not always the right approach for all types of metadata. This method works best when the embedded metadata is meaningful. For example, here we’re showcasing embedding the “title” meta field, which can also provide good context for the embedding model.

Conversational RAG Agent using InMemoryChatMessageStore

Wed, 29 Jul 2026 00:00:00 +0000

Level: Advanced
Time to complete: 20 minutes
Components Used: Agent, ChatPromptBuilder, InMemoryBM25Retriever OpenAIChatGenerator
Archived Components Used: InMemoryChatMessageStore, ChatMessageRetriever, ChatMessageWriter
Prerequisites: You need an OpenAI API Key
Goal: After completing this tutorial, you’ll have learned how to incorporate a conversational history into a RAG pipeline to enable multi-turn conversations grounded in documents.

Overview

In this tutorial, you’ll first build a simple conversational pipeline using chat components and an LLM. You’ll then extend this setup into a conversational RAG pipeline using the Agent component, capable of handling multi-turn interactions over documents.

Serializing LLM Pipelines

Mon, 15 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 10 minutes
Components Used: TransformersChatGenerator, ChatPromptBuilder
Prerequisites: None
Goal: After completing this tutorial, you’ll understand how to serialize and deserialize between YAML and Python code.

Overview

📚 Useful Documentation: Serialization

Serialization means converting a pipeline to a format that you can save on your disk and load later. It’s especially useful because a serialized pipeline can be saved on disk or a database, get sent over a network and more.

Human-in-the-Loop with Haystack Agents

Mon, 20 Jul 2026 00:00:00 +0000

Level: Advanced
Time to complete: 20 minutes
Components Used: Agent, OpenAIChatGenerator
Prerequisites: You need an OpenAI API Key
Goal: After completing this tutorial, you’ll have learned how to implement human-in-the-loop workflows in Haystack agents using confirmation strategies, create custom confirmation policies, and control tool execution approval flows.

Overview

This tutorial introduces how to use confirmation strategies to create human-in-the-loop interactions in Haystack’s Agent component.

Why is this useful? Imagine an AI agent that can access your bank account, send emails, or make purchases. You probably want human approval before it executes sensitive operations. Confirmation strategies let you define exactly when the agent should ask for permission.

Creating Vision+Text RAG Pipelines

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 20 minutes
Components Used: SentenceTransformersDocumentImageEmbedder, ImageFileToDocument, DocumentToImageContent, DocumentTypeRouter, LLMDocumentContentExtractor
Prerequisites: You need an OpenAI API Key
Goal: After completing this tutorial, you’ll have learned how to index and retrieve images using Haystack and build a RAG pipeline that can answer questions grounded in both images and text.

Overview

In this notebook, you’ll learn how to index and retrieve images using Haystack. By the end, you’ll be able to build a Retrieval-Augmented Generation (RAG) pipeline that can answer questions grounded in both images and text. This is useful when working with datasets like scientific papers, diagrams, or screenshots where meaning is spread across modalities.

Creating a Multi-Agent System with Haystack

Fri, 12 Jun 2026 00:00:00 +0000

Level: Advanced
Time to complete: 20 minutes
Components Used: Agent, DuckduckgoApiWebSearch, OpenAIChatGenerator, DocumentWriter
Prerequisites: You need an OpenAI API Key, and a Notion Integration set up beforehand
Goal: After completing this tutorial, you’ll have learned how to build a multi-agent system in Haystack where each agent is specialized for a specific task.

Overview

Multi-agent systems are made up of several intelligent agents that work together to solve complex tasks more effectively than a single agent alone. Each agent takes on a specific role or skill, allowing for distributed reasoning, task specialization, and smooth coordination within one unified system.

Compress the KV Cache with TurboQuant and Haystack

Mon, 15 Jun 2026 00:00:00 +0000

Level: Advanced
Time to complete: 20 min
Components Used: TransformersChatGenerator
Goal: Apply TurboQuant KV cache compression to a local LLM and measure its memory and throughput impact with Haystack.

Overview

Every time an LLM generates a token, it reads and writes a key-value (KV) cache - a growing table of intermediate activations that lets the model attend to previous tokens without recomputing them. On long contexts or large models, this cache becomes the dominant consumer of GPU memory.

Using Pre-Built Agents from Agent Pack

Wed, 22 Jul 2026 00:00:00 +0000

Level: Advanced
Time to complete: 25 minutes
Components/Packages Used: agent-pack-haystack ( create_advanced_rag_agent, create_deep_research_agent), Agent, InMemoryDocumentStore
Prerequisites: Haystack 3.0 or later, an OpenAI API key, and a Tavily API key (free tier available)
Goal: After completing this tutorial, you’ll understand what Agent Pack is and why it exists, and you’ll have run and customized two ready-made agents: the Advanced RAG Agent and the Deep Research Agent.

Overview

A language model and a set of tools are the core building blocks of an agent. But a robust agent usually needs more than that: an architecture that splits work across sub-agents, techniques for keeping the context window small but focused, and hooks to shape the agent loop before and after it runs. Assembling those pieces correctly is where most of the effort goes.

Build an Extractive QA Pipeline

Wed, 17 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 15 minutes
Components Used: TransformersExtractiveReader, InMemoryDocumentStore, InMemoryEmbeddingRetriever, DocumentWriter, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder
Goal: After completing this tutorial, you’ll have learned how to build a Haystack pipeline that uses an extractive model to display where the answer to your query is.

Overview

What is extractive question answering? So glad you asked! The short answer is that extractive models pull verbatim answers out of text. It’s good for use cases where accuracy is paramount, and you need to know exactly where in the text that the answer came from. If you want additional context, here’s a deep dive on extractive versus generative language models.

Creating a Hybrid Retrieval Pipeline

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 15 minutes
Components Used: DocumentSplitter, SentenceTransformersDocumentEmbedder, InMemoryDocumentStore, InMemoryBM25Retriever, InMemoryEmbeddingRetriever, and SentenceTransformersSimilarityRanker
Prerequisites: None
Goal: After completing this tutorial, you will have learned about creating a hybrid retrieval and when it’s useful.

Overview

Hybrid Retrieval combines keyword-based and embedding-based retrieval techniques, leveraging the strengths of both approaches. In essence, dense embeddings excel in grasping the contextual nuances of the query, while keyword-based methods excel in matching keywords.

There are many cases when a simple keyword-based approaches like BM25 performs better than a dense retrieval (for example in a specific domain like healthcare) because a dense model needs to be trained on data. For more details about Hybrid Retrieval, check out Blog Post: Hybrid Document Retrieval.

Generating Structured Output with OpenAI

Mon, 29 Dec 2025 00:00:00 +0000

Level: Beginner
Time to complete: 15 minutes
Prerequisites: You must have an API key from an active OpenAI account as this tutorial uses a GPT model by OpenAI.
Components Used: OpenAIChatGenerator, OpenAIResponsesChatGenerator
Goal: Learn how to generate structured outputs with OpenAIChatGenerator or OpenAIResponsesChatGenerator using Pydantic model or JSON schema.

Overview

This tutorial shows how to produce structured outputs by either providing Pydantic model or JSON schema to OpenAIChatGenerator.

Note: Only latest model starting with gpt-4o-mini can be used for this feature.

Classifying Documents & Queries by Language

Mon, 22 Jun 2026 00:00:00 +0000

Level: Beginner
Time to complete: 15 minutes
Components Used: InMemoryDocumentStore, DocumentLanguageClassifier, MetadataRouter, DocumentWriter, TextLanguageRouter, DocumentJoiner, InMemoryBM25Retriever, ChatPromptBuilder, OpenAIChatGenerator
Goal: After completing this tutorial, you’ll have learned how to build a Haystack pipeline to classify documents based on the (human) language they were written in.
Optionally, at the end you’ll also incorporate language clasification and query routing into a RAG pipeline, so you can query documents based on the language a question was written in.

Overview

In a gobalized society with over 7,000 human languages spoken worldwide today, handling multilingual input is a common use case for NLP applications.

Evaluating RAG Pipelines

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 15 minutes
Components Used: InMemoryDocumentStore, InMemoryEmbeddingRetriever, ChatPromptBuilder, OpenAIChatGenerator, DocumentMRREvaluator, FaithfulnessEvaluator, SASEvaluator
Prerequisites: You must have an API key from an active OpenAI account as this tutorial is using the gpt-4o-mini model by OpenAI: https://platform.openai.com/api-keys
Goal: After completing this tutorial, you’ll have learned how to evaluate your RAG pipelines both with model-based, and statistical metrics available in the Haystack evaluation offering. You’ll also see which other evaluation frameworks are integrated with Haystack.

Overview

In this tutorial, you will learn how to evaluate Haystack pipelines, in particular, Retriaval-Augmented Generation ( RAG) pipelines.

Building an Agentic RAG with Fallback to Websearch

Fri, 12 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 10 minutes
Components Used: ConditionalRouter, SerperDevWebSearch, ChatPromptBuilder, OpenAIChatGenerator
Prerequisites: You must have an OpenAI API Key and a Serper API Key for this tutorial
Goal: After completing this tutorial, you’ll have learned how to create an agentic RAG pipeline with conditional routing that can fallback to websearch if the answer is not present in your dataset.

Overview

When developing applications using retrieval augmented generation ( RAG), the retrieval step plays a critical role. It serves as the primary information source for large language models (LLMs) to generate responses. However, if your database lacks the necessary information, the retrieval step’s effectiveness is limited. In such scenarios, it may be practical incorporate agentic behavior and use the web as a fallback data source for your RAG application. By implementing a conditional routing mechanism in your system, you gain complete control over the data flow, enabling you to design a system that can leverage the web as its data source under some conditions.

Building a Chat Agent with Function Calling

Sat, 11 Jul 2026 00:00:00 +0000

Level: Advanced
Time to complete: 20 minutes
Components Used: InMemoryDocumentStore, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, InMemoryEmbeddingRetriever, ChatPromptBuilder, OpenAIChatGenerator, Agent
Prerequisites: You must have an OpenAI API Key and be familiar with creating pipelines
Goal: After completing this tutorial, you will have learned how to build chat applications that demonstrate agent-like behavior using OpenAI’s function calling feature.

Overview

📚 Useful Sources:

OpenAI’s function calling connects large language models to external tools. By providing a tools list with functions and their specifications to the OpenAI API calls, you can easily build chat assistants that can answer questions by calling external APIs or extract structured information from text.

Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter

Wed, 17 Jun 2026 00:00:00 +0000

Level: Intermediate
Time to complete: 15 minutes
Components Used: TransformersZeroShotTextRouter, TransformersTextRouter, InMemoryDocumentStore, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, InMemoryEmbeddingRetriever, InMemoryBM25Retriever
Goal: After completing this tutorial, you will have learned about TransformersZeroShotTextRouter and TransformersTextRouter and how to use them in a pipeline.

Overview

One of the great benefits of using state-of-the-art NLP models like those available in Haystack is that it allows users to state their queries as plain natural language questions: rather than trying to come up with just the right set of keywords to find the answer to their question, users can simply ask their question in much the same way that they would ask it of a (very knowledgeable!) person.

Retrieving a Context Window Around a Sentence

Fri, 30 Jan 2026 00:00:00 +0000

Level: Beginner
Time to complete: 10 minutes
Components Used: SentenceWindowRetriever, DocumentSplitter, InMemoryDocumentStore, InMemoryBM25Retriever
Goal: After completing this tutorial, you will have learned about Sentence-Window Retrieval and how to use it for document retrieval.

Overview

The Sentence-Window retrieval technique is a simple and effective way to retrieve more context given a user query which matched some document. It is based on the idea that the most relevant sentences are likely to be close to each other in the document. The technique involves selecting a window of sentences around a sentence matching a user query and instead of returning the matching sentence, the entire window is returned. This technique can be particularly useful when the user query is a question or a phrase that requires more context to be understood.

Evaluation

Wed, 29 Jul 2026 00:00:00 +0000

Evaluation measures performance using metrics like precision, recall, and relevancy, providing a clear picture of your pipeline’s strengths and weaknesses using LLMs or ground-truth labels.

Evaluating RAG systems can help understand performance bottlenecks and optimize one component at a time, for example, a Retriever or a prompt used with a Generator.

Here’s a step-by-step guide explaining what you need to evaluate, how you evaluate, and how you can improve your application after evaluation using Haystack!

Hybrid RAG Pipeline with Breakpoints

Sat, 01 Aug 2026 00:00:00 +0000

This notebook demonstrates how to setup breakpoints in a Haystack pipeline. In this case, we will set up break points in a hybrid retrieval-augmented generation (RAG) pipeline. The pipeline combines BM25 and embedding-based retrieval methods, then uses a transformer-based reranker and an LLM to generate answers.

Install packages

%%bash

pip install haystack-ai>=2.16.0
pip install "transformers[torch,sentencepiece]"
pip install sentence-transformers-haystack

Setup OpenAI API keys

import os
from getpass import getpass

# Enable writing pipeline snapshots to disk when a breakpoint is hit
os.environ["HAYSTACK_PIPELINE_SNAPSHOT_SAVE_ENABLED"] = "true"

if "OPENAI_API_KEY" not in os.environ:
    os.environ["OPENAI_API_KEY"] = getpass("Enter OpenAI API key:")

Import Required Libraries

First, let’s import all the necessary components from Haystack.

Advanced RAG: Query Expansion

Wed, 29 Jul 2026 00:00:00 +0000

by Tuana Celik ( LI, Twitter/X)

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval cookbook & full article

2️⃣ Query Expansion & the full article

3️⃣ Query Decomposition cookbook & full article

4️⃣ Automated Metadata Enrichment

In this cookbook, you’ll learn how to implement query expansion for RAG. Query expansion consists of asking an LLM to produce a number of similar queries to a user query. We are then able to use each of these queries in the retrieval process, increasing the number and relevance of retrieved documents.

Calculating a Hallucination Score with the OpenAIChatGenerator

Wed, 29 Jul 2026 00:00:00 +0000

In this cookbook we will show how to calculate a hallucination risk based on the research paper LLMs are Bayesian, in Expectation, not in Realization and this GitHub repo, https://github.com/leochlon/hallbayes.

In this notebook, we’ll use the OpenAIChatGenerator from haystack-experimental.

⚠️ Archived package: haystack-experimental is archived and no longer maintained. This experiment was not adopted into Haystack core, so this notebook pins the final release, haystack-experimental==0.19.0.post1. It stays installable from PyPI, but is only tested against the version of Haystack that was current in February 2026.

Computer-Use Agent with Skills

Thu, 23 Jul 2026 00:00:00 +0000

Notebook by Kacper Łukawski

🚀 Part of Haystack 3.0 Launch Week: five days of new drops (July 20–24).

In this notebook, we build a fully local, fully async agent that uses a skill to change how it reports back, and a custom bash tool to actually use the machine it runs on.

Haystack’s Agent can be given skills - folders of instructions the agent reads on demand, in the same SKILL.md format used by Claude Code and Codex. A skill teaches the agent how to do something. It doesn’t let the agent do anything by itself: skills teach, tools do.

Building a Cost-Aware Agent with Hooks

Wed, 22 Jul 2026 00:00:00 +0000

Notebook by Bilge Yücel

🚀 Part of Haystack 3.0 Launch Week: five days of new drops (July 20–24).

Every call to Agent.run() returns metadata alongside the agent’s reply. In this cookbook you’ll use that metadata, and Agent hooks, to enforce soft and hard budget policies.

You’ll:

Build a simple agent with a custom tool and inspect step_count, token_usage, and tool_call_counts.
Implement a reusable post-run budget policy with soft and hard limits.
Enforce the same limits inside the agent loop with hooks, so an over-budget run can stop before the next LLM call.

Prerequisite: An OpenAI API key.

Haystack 3.0.0

Mon, 20 Jul 2026 00:00:00 +0000

⭐️ Highlights

Haystack 3.0 is a major release for building production-grade agents with full control and flexibility. It ships a wave of new capabilities: a more capable Agent with hooks and first-class skills, built-in run introspection, first-class async for serving, a leaner core, and safer pipeline loading.

A few small, intentional breaking changes come with it but our Migration Guide and Upgrading to Haystack 3.0 make it easy to upgrade.

⚠️ marks a breaking change. See Upgrade Notes below for migration details.

Introducing Haystack 3.0: Agent Hooks, Skills and a Lighter Core

Mon, 20 Jul 2026 00:00:00 +0000

Today, we’re happy to announce the release of a new major version: Haystack 3.0 🎉

Haystack is an open-source AI orchestration framework for building production-grade agents with full control and flexibility, and 3.0 is the release where agents move to the center of the framework. On top of a lighter core, this release ships a wave of agentic capabilities: first-class skills, hooks to control the agent loop, built-in run introspection, and high-level, pre-built agents for common tasks such as deep research.

Haystack 2.31.0

Wed, 08 Jul 2026 00:00:00 +0000

⭐️ Highlights

📦 Slimming down Haystack core ahead of 3.0

This release begins the migration of many components out of haystack core and into dedicated integration packages, in preparation for Haystack 3.0. Components with heavy or optional dependencies — including all SentenceTransformers embedders and rankers, the Hugging Face API components, the legacy Generators, TikaDocumentConverter, AzureOCRDocumentConverter, the Whisper transcribers, the OpenAPI connectors, the spaCy and Transformers extractors/classifiers/routers, SerperDevWebSearch, SearchApiWebSearch, DocumentLanguageClassifier/TextLanguageRouter, and the Datadog and OpenTelemetry tracers — are now deprecated and will be removed in 3.0.

MCP + Haystack: A Practical Guide for AI Engineers

Mon, 22 Jun 2026 00:00:00 +0000

The Model Context Protocol (MCP) is the connective tissue of the modern AI stack. If you are building AI agents or production RAG systems, understanding how MCP works with Haystack is one of the most direct ways to make your applications composable, reusable, and reachable from the tools your users already live in.

This blog post explains what MCP is, why it matters, and the different ways you can use MCP with Haystack, whether you want to consume external tools inside a Haystack agent or expose your Haystack pipelines and agents as MCP tools for MCP clients like Claude, ChatGPT, and Cursor.

Multilingual RAG on a Podcast

Fri, 19 Jun 2026 00:00:00 +0000

Notebook by Stefano Fiorucci

This notebook shows how to create a multilingual Retrieval Augmented Generation application, starting from a podcast.

🧰 Stack:

Haystack LLM framework
OpenAI Whisper model for audio transcription
Qdrant vector database
multilingual embedding model: multilingual-e5-large
multilingual LLM: Mistral Small

Installation

%%capture
! pip install -U mistral-haystack haystack-ai qdrant-haystack whisper-haystack "openai-whisper>=20231106" pytube sentence-transformers-haystack "huggingface_hub>=0.23.0"

Podcast transcription

download the audio from Youtube using pytube
transcribe it locally using Haystack’s LocalWhisperTranscriber with the whisper-small model. We could use bigger models, which take longer to transcribe. We could also call the paid OpenAI API, using RemoteWhisperTranscriber.

Since the transcription takes some time (about 10 minutes), I commented out the following code and will provide the transcription.

Haystack 2.30.2

Thu, 18 Jun 2026 00:00:00 +0000

🐛 Bug Fixes

Fixed the Agent exiting prematurely under the default exit_conditions=["text"]. The agent now only stops when the last message is an assistant message with non-empty text (or when no tool invoker is configured). Previously, if the LLM produced an invalid tool call that was discarded, the resulting assistant message with empty text and no tool calls would trigger an exit, preventing the agent from recovering. The agent now continues looping so the model can recover on the next iteration.

Extract Metadata Filters from a Query

Wed, 17 Jun 2026 00:00:00 +0000

Notebook by David Batista

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval & the full article

2️⃣ Query Expansion cookbook & full article

3️⃣ Query Decomposition cookbook & the full article

4️⃣ Automated Metadata Enrichment

In this notebook, we’ll discuss how to implement a custom component, QueryMetadataExtractor, that extracts entities from the query and formulates the corresponding metadata filter.

Useful Sources

Setup the Development Environment

!pip install haystack-ai

Enter your OPENAI_API_KEY. Get your OpenAI API key here:

Extracting Metadata with an LLM

Wed, 17 Jun 2026 00:00:00 +0000

Notebook by David S. Batista

This notebook shows how to use LLMMetadataExtractor, we will use a arge Language Model to perform metadata extraction from a Document.

Setting Up

!uv pip install haystack-ai
!uv pip install sentence-transformers-haystack

Initialize LLMMetadataExtractor

Let’s define what kind of metadata we want to extract from our documents, we wil do it through a LLM prompt, which will then be used by the LLMMetadataExtractor component. In this case we want to extract named-entities from our documents.

Improve Retrieval by Embedding Meaningful Metadata

Wed, 17 Jun 2026 00:00:00 +0000

Notebook by Stefano Fiorucci

In this notebook, I do some experiments on embedding meaningful metadata to improve Document retrieval.

%%capture
! pip install wikipedia haystack-ai sentence-transformers-haystack rich

import rich

Load data from Wikipedia

We are going to download the Wikipedia pages related to some bands, using the python library wikipedia.

These pages are converted into Haystack Documents.

some_bands="""The Beatles
Rolling stones
Dire Straits
The Cure
The Smiths""".split("\n")

import wikipedia
wikipedia.set_user_agent("haystack-cookbook (https://github.com/deepset-ai/haystack-cookbook)")
from haystack.dataclasses import Document

raw_docs=[]

for title in some_bands:
    page = wikipedia.page(title=title, auto_suggest=False)
    doc = Document(content=page.content, meta={"title": page.title, "url":page.url})
    raw_docs.append(doc)

🔧 Setup the experiment

Utility functions to create Pipelines

The indexing Pipeline transforms the Documents and stores them (with vectors) in a Document Store. The retrieval Pipeline takes a query as input and perform the vector search.

Prompt Optimization with DSPy

Wed, 17 Jun 2026 00:00:00 +0000

When building applications with LLMs, writing effective prompts is a long process of trial and error. Often, if you switch models, you also have to change the prompt. What if you could automate this process?

That’s where DSPy comes in - a framework designed to algorithmically optimize prompts for Language Models. By applying classical machine learning concepts (training and evaluation data, metrics, optimization), DSPy generates better prompts for a given model and task.

Running Haystack Pipelines in Asynchronous Environments

Wed, 17 Jun 2026 00:00:00 +0000

Notebook by Madeeswaran Kannan & Mathis Lucka

In this notebook, you’ll learn how to use the AsyncPipeline and async-enabled components to build and execute a Haystack pipeline in an asynchronous environment. It’s based on this short Haystack tutorial, so it would be a good idea to familiarize yourself with it before we begin. A further prerequisite is working knowledge of cooperative scheduling and async programming in Python.

Motivation

By default, the Pipeline class in haystack is a regular Python object class that exposes non-async methods to add/connect components and execute the pipeline logic. Currently, it can be used in async environments, but it’s not optimal to do so since it executes its logic in a ‘ blocking’ fashion, i.e., once the Pipeline.run method is invoked, it must run to completion and return the outputs before the next statement of code can be executed¹. In a typical async environment, this prevents active async event loop from scheduling other async coroutines, thereby reducing throughput. To mitigate this bottleneck, we introduce the concept of async-enabled Haystack components and an AsyncPipeline class that cooperatively schedules the execution of both async and non-async components.

Using Hypothetical Document Embeddings (HyDE) to Improve Retrieval

Wed, 17 Jun 2026 00:00:00 +0000

📚 This cookbook has an accompanying article with a complete walkthrough “Optimizing Retrival with HyDE”

In this coookbook, we are building Haystack components that allow us to easily incorporate HyDE into our RAG pipelines, to optimize retrieval.

To learn more about HyDE and when it’s useful, check out our guide to Hypothetical Document Embeddings (HyDE)

Install Requirements

!pip install haystack-ai sentence-transformers-haystack datasets

In the following sections, we will be using the OpenAIChatGenerator, so we need to provide our API key 👇

Web QA with Mistral

Wed, 17 Jun 2026 00:00:00 +0000

Colab by Tuana Celik - ( LI & Twitter)

Quick guide to building Question Answering on the web with a Mistral AI model and Haystack. We use mistral-small-latest via the Mistral AI API and the MistralChatGenerator from the official Mistral integration for Haystack.

Use the MistralChatGenerator to query the model on its own
Add the generator to a full RAG Pipeline (on the web)

Install dependencies

!uv pip install haystack-ai trafilatura sentence-transformers-haystack mistral-haystack

Prompt the Model - Standalone

We are using the Mistral AI API with mistral-small-latest.

Advanced Prompt Customization for Anthropic

Tue, 16 Jun 2026 00:00:00 +0000

Notebook by Bilge Yucel ( LI & X (Twitter))

In this example, we’ll create a RAG application using prompting techniques in Anthropic’s Prompt Engineering Guide. This application will use Anthropic Claude 3 models and Haystack to extract relevant quotes from given documents and generate an answer based on extracted quotes.

📚 Useful Sources:

Setup the Development Environment

Install antropic-haystack package and other required packages with pip:

!pip install anthropic-haystack "datasets>=2.6.1" sentence-transformers-haystack

You need an ANTHROPIC_API_KEY to work with Claude models. Get your API key here

Agentic RAG with Llama 3.2 3B

Tue, 16 Jun 2026 00:00:00 +0000

In their Llama 3.2 collection, Meta released two small yet powerful Language Models.

In this notebook, we’ll use the 3B model to build an Agentic Retrieval Augmented Generation application.

🎯 Our goal is to create a system that answers questions using a knowledge base focused on the Seven Wonders of the Ancient World. If the retrieved documents don’t contain the answer, the application will fall back to web search for additional context.

AstraDB 🤝 Haystack Integration

Tue, 16 Jun 2026 00:00:00 +0000

In this notebook, you’ll learn how to use AstraDB as a data source in your Haystack pipelines.

Prerequisites

You’ll need an OpenAPI key to follow along. (Haystack is model-agnostic so feel free to use a different one if you’d prefer!)

You’ll need the following variables in order to use the Haystack extension. The following tutorials will show you how to create an AstraDB database, and save these pieces of information.

API Endpoint
Token
Astra keyspace
Astra collection name

Follow the first step in this this tutorial to create a free AstraDB database and save your database ID, application token, keyspace, and database region.

Breakpoint on Agent in a Pipeline

Tue, 16 Jun 2026 00:00:00 +0000

This notebook demonstrates how to set up breakpoints within an Agent component in a Haystack pipeline. Breakpoints can be placed either on the chat_generator or on any of the tools used by the Agent. This guide showcases both approaches.

The pipeline features an Agent acting as a database assistant, responsible for extracting relevant information and writing it to the database.

Install packages

%%bash

pip install "haystack-ai>=2.16.1"
pip install "transformers[torch,sentencepiece]"

Setup OpenAI API key for the chat_generator

Define & Run Tools

Tue, 16 Jun 2026 00:00:00 +0000

In this notebook, we introduce the features we’ve developed for tool/function calling support in Haystack.

We refactored the ChatMessage dataclass, to be more flexible and future-proof.
We introduced some new dataclasses: ToolCall, ToolCallResult, and Tool.
We added support for tools in the OpenAIChatGenerator and other Chat Generators.
We introduced the ToolInvoker component, to actually execute tool calls prepared by Language Models.

We will first introduce the new features and then show two examples:

Evaluating AI with Haystack

Tue, 16 Jun 2026 00:00:00 +0000

by Bilge Yucel ( X, Linkedin)

In this cookbook, we walk through the Evaluators in Haystack, create an evaluation pipeline and try different Evaluation Frameworks like FlowJudge.

📚 Useful Resources:

📺 Watch Along

!pip install haystack-ai sentence-transformers-haystack pypdf "flow-judge[hf]"

1. Building your pipeline

ARAGOG

This dataset is based on the paper Advanced Retrieval Augmented Generation Output Grading (ARAGOG). It’s a collection of papers from ArXiv covering topics around Transformers and Large Language Models, all in PDF format.

Function Calling and Multimodal QA with Gemini

Tue, 16 Jun 2026 00:00:00 +0000

by Tuana Celik: Twitter, LinkedIn, Tilde Thurium: Twitter, LinkedIn and Silvano Cerza: LinkedIn

This is a notebook showing how you can use Gemini + Vertex AI with Haystack.

To use Gemini models on the Gemini Developer API with Haystack, check out our documentation.

Gemini is Google’s newest model. You can read more about its capabilities here.

Install dependencies

As a prerequisite, you need to have a Google Cloud Project set up that has access to Vertex AI and Gemini.

RAG Evaluation with Prometheus 2

Tue, 16 Jun 2026 00:00:00 +0000

Evaluating the responses of Language Models and LLM-based applications often involves using model-based metrics that do not require ground truth labels. Large proprietary models like GPT-4 and Claude 3 Opus are frequently employed as evaluators and demonstrate a good correlation with human evaluations.

However, relying on closed models poses several challenges:

fairness: the training data of these models is unknown.
controllability: the behavior of these models can change unpredictably.
data privacy: sending data to external providers may raise privacy concerns.
affordability: using these powerful models can be expensive.

Using open models for evaluation is an active research area, but their practical use is often limited. They typically do not correlate well with human judgments and lack flexibility.

RAG with Llama 3.1

Tue, 16 Jun 2026 00:00:00 +0000

Simple RAG example on the Oscars using Llama 3.1 open models and the Haystack LLM framework.

Installation

! pip install transformers-haystack sentence-transformers-haystack bitsandbytes

Authorization

you need an Hugging Face account
you need to accept Meta conditions here: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct and wait for the authorization

import getpass, os


os.environ["HF_API_TOKEN"] = getpass.getpass("Your Hugging Face token")

Your Hugging Face token··········

RAG with Llama-3.1-8B-Instruct (about the Oscars) 🏆🎬

! pip install wikipedia

Load data from Wikipedia

from IPython.display import Image
from pprint import pprint
import rich
import random

import wikipedia
wikipedia.set_user_agent("haystack-cookbook (https://github.com/deepset-ai/haystack-cookbook)")
wikipedia.set_rate_limiting(True)
from haystack.dataclasses import Document

title = "96th_Academy_Awards"
page = wikipedia.page(title=title, auto_suggest=False)
raw_docs = [Document(content=page.content, meta={"title": page.title, "url":page.url})]

Indexing Pipeline

from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack import Document
from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder
from haystack.components.converters import TextFileToDocument
from haystack.components.writers import DocumentWriter
from haystack.components.preprocessors import DocumentSplitter
from haystack.utils import ComponentDevice

document_store = InMemoryDocumentStore()

indexing_pipeline = Pipeline()
indexing_pipeline.add_component("splitter", DocumentSplitter(split_by="word", split_length=200))

indexing_pipeline.add_component(
    "embedder",
    SentenceTransformersDocumentEmbedder(
        model="Snowflake/snowflake-arctic-embed-l",  # good embedding model: https://huggingface.co/Snowflake/snowflake-arctic-embed-l
        device=ComponentDevice.from_str("cuda:0"),    # load the model on GPU
    ))
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))

# connect the components
indexing_pipeline.connect("splitter", "embedder")
indexing_pipeline.connect("embedder", "writer")

<haystack.core.pipeline.pipeline.Pipeline object at 0x7fcc409ea4d0>
🚅 Components
  - splitter: DocumentSplitter
  - embedder: SentenceTransformersDocumentEmbedder
  - writer: DocumentWriter
🛤️ Connections
  - splitter.documents -> embedder.documents (List[Document])
  - embedder.documents -> writer.documents (List[Document])

indexing_pipeline.run({"splitter":{"documents":raw_docs}})

/usr/local/lib/python3.10/dist-packages/sentence_transformers/SentenceTransformer.py:174: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v3 of SentenceTransformers.
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:81: UserWarning: 
Access to the secret `HF_TOKEN` has not been granted on this notebook.
You will not be requested again.
Please restart the session if you want to be prompted again.
  warnings.warn(



Batches:   0%|          | 0/1 [00:00<?, ?it/s]





{'writer': {'documents_written': 12}}

RAG Pipeline

from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

template = [ChatMessage.from_user("""
Using the information contained in the context, give a comprehensive answer to the question.
If the answer cannot be deduced from the context, do not give an answer.

Context:
  {% for doc in documents %}
  {{ doc.content }} URL:{{ doc.meta['url'] }}
  {% endfor %};
  Question: {{query}}

""")]
prompt_builder = ChatPromptBuilder(template=template)

Here, we use the TransformersChatGenerator, loading the model in Colab with 4-bit quantization.

Speaker Diarization with AssemblyAI

Tue, 16 Jun 2026 00:00:00 +0000

📚 This cookbook has an accompanying article with a complete walkthrough “ Level up Your RAG Application with Speaker Diarization”

LLMs excel with text data, answering complex questions without manual reading or searching. When dealing with audio or video, providing transcription is key. Transcription captures spoken content of the audio or video, but in multi-speaker recordings, it may miss non-verbal information and fail to convey speaker count or individual remarks. Therefore, to maximize the LLM’s potential with such recordings, Speaker Diarization is essential!

🪁 RAG pipelines with Haystack + Zephyr 7B Beta

Tue, 16 Jun 2026 00:00:00 +0000

Notebook by Stefano Fiorucci and Tuana Celik

We are going to build a nice Retrieval Augmented Generation pipeline for Rock music, using the 🏗️ Haystack LLM orchestration framework and a good LLM: 💬 Zephyr 7B Beta (fine-tuned version of Mistral 7B V.01 that focuses on helpfulness and outperforms many larger models on the MT-Bench and AlpacaEval benchmarks)

Install dependencies

wikipedia is needed to download data from Wikipedia
haystack-ai is the Haystack package
sentence_transformers is needed for embeddings
transformers is needed to use open-source LLMs
accelerate and bitsandbytes are required to use quantized versions of these models (with smaller memory footprint)

%%capture
! pip install wikipedia transformers-haystack bitsandbytes sentence-transformers-haystack

from IPython.display import Image
from pprint import pprint
import torch
import rich
import random

Load data from Wikipedia

We are going to download the Wikipedia pages related to some Rock bands, using the python library wikipedia.

Web-Enhanced Self-Reflecting Agent

Wed, 10 Jun 2026 00:00:00 +0000

Notebook by Bilge Yucel

In this notebook, we will use Ollama, Gemma2 and Haystack to build a self-reflecting agent that can leverage web resources to augment its self-reflection and decision-making capabilities.

📚 Useful Sources

Install Dependencies

!pip install -U ollama-haystack serperdev-haystack trafilatura 'nltk>=3.9.1' git+https://github.com/deepset-ai/haystack.git@main

Enable Tracing

import logging
from haystack import tracing
from haystack.tracing.logging_tracer import LoggingTracer

logging.basicConfig(format="%(levelname)s - %(name)s -  %(message)s", level=logging.WARNING)
logging.getLogger("haystack").setLevel(logging.DEBUG)

tracing.tracer.is_content_tracing_enabled = True # to enable tracing/logging content (inputs/outputs)
tracing.enable_tracing(LoggingTracer(tags_color_strings={"haystack.component.input": "\x1b[1;31m", "haystack.component.name": "\x1b[1;34m"}))

Disable Tracing

tracing.tracer.is_content_tracing_enabled = False
tracing.disable_tracing()

Index Documents for the Agent

from haystack import Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.writers import DocumentWriter
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter

document_store = InMemoryDocumentStore()

# Indexing pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component(instance=LinkContentFetcher(), name="fetcher")
indexing_pipeline.add_component(instance=HTMLToDocument(), name="converter")
indexing_pipeline.add_component(instance=DocumentCleaner(), name="cleaner")
indexing_pipeline.add_component(instance=DocumentSplitter(split_by="sentence", split_length=5, split_overlap=1), name="splitter")
indexing_pipeline.add_component(instance=DocumentWriter(document_store = document_store), name="writer")

indexing_pipeline.connect("fetcher.streams", "converter.sources")
indexing_pipeline.connect("converter.documents", "cleaner")
indexing_pipeline.connect("cleaner", "splitter")
indexing_pipeline.connect("splitter", "writer.documents")

# index some documentation pages to use for RAG
indexing_pipeline.run({
    "fetcher": {
        "urls": [
            "https://docs.haystack.deepset.ai/docs/choosing-the-right-generator",
            "https://docs.haystack.deepset.ai/docs/ollamagenerator",
            "https://haystack.deepset.ai/overview/quick-start",
            "https://haystack.deepset.ai/overview/intro"
            ]}})

/Users/bilgeyucel/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(





{'writer': {'documents_written': 70}}

API Keys

import os

os.environ["SERPERDEV_API_KEY"] = "***"

Build the Agent

Routes

from haystack.components.routers import ConditionalRouter

main_routes = [
    {
        "condition": "{{'N0_ANSWER' in replies[0].replace('\n', '')}}",
        "output" :"{{query}}",
        "output_name": "go_web",
        "output_type": str,
    },
    {
        "condition": "{{'NO_ANSWER' not in replies[0].replace('\n', '')}}",
        "output": "{{replies[0]}}",
        "output_name": "answer",
        "output_type": str,
    },
]

Agent Prompt

agent_prompt_template = """<start_of_turn>user
{% if web_documents %}
    You were asked to answer the following query given the documents retrieved from Haystack's documentation but the context was not enough.
    Here is the user question: {{ query }}
    Context:
    {% for document in documents %}
        {{document.content}}
    {% endfor %}
    {% for document in web_documents %}
    URL: {{document.meta.link}}
    TEXT: {{document.content}}
    ---
    {% endfor %}
    Answer the question based on the given context.
    If you have enough context to answer this question, return your answer with the used links.
    If you don't have enough context to answer, say 'N0_ANSWER'.
{% else %}
Answer the following query based on the documents retrieved from Haystack's documentation.

Documents:
{% for document in documents %}
  {{document.content}}
{% endfor %}

Query: {{query}}

If you have enough context to answer this question, just return your answer
If you don't have enough context to answer, say 'N0_ANSWER'.
{% endif %}

<end_of_turn>
<start_of_turn>model
"""

Web-Enhanced Self-Reflecting Agent

from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack_integrations.components.websearch.serperdev import SerperDevWebSearch
from haystack.components.builders import PromptBuilder
from haystack_integrations.components.generators.ollama import OllamaGenerator

self_reflecting_agent = Pipeline(max_runs_per_component=5)
self_reflecting_agent.add_component("retriever", InMemoryBM25Retriever(document_store=document_store, top_k=3))
self_reflecting_agent.add_component("prompt_builder_for_agent", PromptBuilder(template=agent_prompt_template))
self_reflecting_agent.add_component("llm_for_agent", OllamaGenerator(model="gemma2:9b-instruct-q4_1", url = "http://localhost:11434"))
self_reflecting_agent.add_component("web_search", SerperDevWebSearch())
self_reflecting_agent.add_component("router", ConditionalRouter(main_routes))

self_reflecting_agent.connect("retriever.documents", "prompt_builder_for_agent.documents")
self_reflecting_agent.connect("prompt_builder_for_agent", "llm_for_agent")
self_reflecting_agent.connect("llm_for_agent.replies", "router.replies")
self_reflecting_agent.connect("router.go_web", "web_search.query")
self_reflecting_agent.connect("web_search.documents", "prompt_builder_for_agent.web_documents")

<haystack.core.pipeline.pipeline.Pipeline object at 0x286bf0f40>
🚅 Components
  - retriever: InMemoryBM25Retriever
  - prompt_builder_for_agent: PromptBuilder
  - llm_for_agent: OllamaGenerator
  - web_search: SerperDevWebSearch
  - router: ConditionalRouter
🛤️ Connections
  - retriever.documents -> prompt_builder_for_agent.documents (List[Document])
  - prompt_builder_for_agent.prompt -> llm_for_agent.prompt (str)
  - llm_for_agent.replies -> router.replies (List[str])
  - web_search.documents -> prompt_builder_for_agent.web_documents (List[Document])
  - router.go_web -> web_search.query (str)

query = "What is Haystack?"
result = self_reflecting_agent.run({"retriever":{"query":query}, "prompt_builder_for_agent":{"query":query}, "router":{"query":query}}, include_outputs_from={"retriever", "router", "llm_for_agent", "web_search", "prompt_builder_for_agent"})

/Users/bilgeyucel/Library/Python/3.9/lib/python/site-packages/haystack/core/pipeline/pipeline.py:521: RuntimeWarning: Pipeline is stuck running in a loop. Partial outputs will be returned. Check the Pipeline graph for possible issues.
  warn(RuntimeWarning(msg))

print(result["router"]["answer"])

Haystack is an open source framework for building production-ready LLM applications, retrieval-augmented generative pipelines and state-of-the-art search systems that work intelligently over large document collections. It lets you quickly try out the latest AI models while being flexible and easy to use. Our inspiring community of users and builders has helped shape Haystack into the modular, intuitive, complete framework it is today.

query = "What's Gemma2?"
result= self_reflecting_agent.run({"retriever":{"query":query}, "prompt_builder_for_agent":{"query":query}, "router":{"query":query}}, include_outputs_from={"retriever", "router", "llm_for_agent", "web_search", "prompt_builder_for_agent"})

print(result["router"]["answer"])

Gemma 2 is Google's latest iteration of open LLMs. It comes in two sizes, 9 billion and 27 billion parameters with base (pre-trained) and instruction-tuned versions.  

**Used Links:**
https://huggingface.co/blog/gemma2

Haystack 2.30.1

Tue, 09 Jun 2026 00:00:00 +0000

⚡️ Enhancement Notes

AzureOpenAIChatGenerator now accepts a Secret for the azure_endpoint and api_version parameters in addition to a plain string. This makes it possible to resolve these values from environment variables at runtime, for example with Secret.from_env_var("AZURE_OPENAI_ENDPOINT"), so the same serialized pipeline can switch between environments (e.g. dev and prod) by changing environment variables instead of the pipeline definition.

AI Guardrails: Content Moderation and Safety with Open Language Models

Fri, 05 Jun 2026 00:00:00 +0000

Deploying safe and responsible AI applications requires robust guardrails to detect and handle harmful, biased, or inappropriate content. In response to this need, several open Language Models have been specifically trained for content moderation, toxicity detection, and safety-related tasks.

This notebook focuses on generative Language Models. Unlike traditional classifiers that output probabilities for predefined labels, generative models produce natural language outputs, even when used for classification tasks.

To support these use cases in Haystack, we’ve introduced the LLMMessagesRouter, a component that routes Chat Messages based on safety classifications provided by a generative Language Model.

Legal Document Analysis with jina-embeddings-v2-base-en

Fri, 05 Jun 2026 00:00:00 +0000

One foggy day in October 2023, I was narrowly excused from jury duty. I had mixed feelings about it, since it actually seemed like a pretty interesting case (Google v. Sonos). A few months later, I idly wondered how the proceedings turned out. I could just read the news, but what’s the fun in that? Let’s see how AI can solve this problem.

Jina.ai recently released jina-embeddings-v2-base-en. It’s an open-source text embedding model capable of accommodating up to 8192 tokens. Splitting text into larger chunks is helpful for understanding longer documents. One of the use cases this model is especially suited for is legal document analysis.

RAG Pipeline Using FastEmbed for Embeddings Generation

Fri, 05 Jun 2026 00:00:00 +0000

FastEmbed is a lightweight, fast, Python library built for embedding generation, maintained by Qdrant. It is suitable for generating embeddings efficiently and fast on CPU-only machines.

In this notebook, we will use FastEmbed-Haystack integration to generate embeddings for indexing and RAG.

Haystack Useful Sources

Install dependencies

!pip install fastembed-haystack qdrant-haystack wikipedia transformers huggingface-api-haystack

Download contents and create docs

favourite_bands="""Audioslave
Green Day
Muse (band)
Foo Fighters (band)
Nirvana (band)""".split("\n")

import wikipedia
wikipedia.set_user_agent("haystack-tutorials")
from haystack.dataclasses import Document

raw_docs=[]
for title in favourite_bands:
    page = wikipedia.page(title=title, auto_suggest=False)
    doc = Document(content=page.content, meta={"title": page.title, "url":page.url})
    raw_docs.append(doc)

Clean, split and index documents on Qdrant

from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter
from haystack_integrations.components.embedders.fastembed import FastembedDocumentEmbedder
from haystack.document_stores.types import DuplicatePolicy

document_store = QdrantDocumentStore(
    ":memory:",
    embedding_dim =384,
    recreate_index=True,
    return_embedding=True,
    wait_result_from_api=True,
)

cleaner = DocumentCleaner()
splitter = DocumentSplitter(split_by='period', split_length=3)
splitted_docs = splitter.run(cleaner.run(raw_docs)["documents"])

len(splitted_docs["documents"])

FastEmbed Document Embedder

Here we are initializing the FastEmbed Document Embedder and using it to generate embeddings for the documents. We are using a small and good model, BAAI/bge-small-en-v1.5 and specifying the parallel parameter to 0 to use all available CPU cores for embedding generation.

Streaming Model Explorer

Fri, 05 Jun 2026 00:00:00 +0000

notebook by Tilde Thurium: Mastodon || Twitter || LinkedIn

Problem: there are so many LLMs these days! Which model is the best for my use case?

This notebook uses Haystack to compare the results of sending the same prompt to several different models.

This is a very basic demo where you can only compare a few models that support streaming responses. I’d like to support more models in the future, so watch this space for updates.

Haystack 2.30.0

Wed, 03 Jun 2026 00:00:00 +0000

⭐️ Highlights

🐍 Syntax-aware Python code splitting with `PythonCodeSplitter`

The new PythonCodeSplitter is a syntax-aware splitter for Python source files, built for code-RAG and code-search pipelines where naive line-based splitting tends to cut through functions and lose structural context. It parses sources with the ast module and greedily merges units, such as module docstring, import blocks, top-level functions, class headers, methods, and nested classes, into chunks of roughly max_effective_lines, keeping whole functions and methods together. For functions that exceed oversized_factor * max_effective_lines, it falls back to a line-based secondary split with overlap.

Using Mem0 Memory Store with Haystack Agents

Tue, 26 May 2026 00:00:00 +0000

Mem0 is a managed memory layer for AI agents. Instead of passing entire conversation histories to an LLM on every turn, Mem0 intelligently extracts and compresses key facts from conversations into optimized memory representations.

At a high level, Mem0 manages a cycle of extraction, consolidation, and retrieval. When new messages arrive, relevant facts are identified and stored. Over time, memories are merged, updated, or allowed to fade if they lose relevance. When the agent later needs context, Mem0 surfaces only the memories most relevant to the current query, helping to keep token usage and latency low.

Agentic Itinerary Planning with OpenStreetMap

Fri, 15 May 2026 00:00:00 +0000

Notebook by Grexrr

OpenStreetMap is a free, community-driven map of the world. In this notebook, we use the osm-integration-haystack package to turn OpenStreetMap data into Haystack Documents and then plug them into LLM workflows.

We’ll together walk through two progressively more advanced scenarios:

Basic OSM query → LLM summarization
Use OSMFetcher to retrieve and preprocess nearby points of interest (POIs) around Cork city centre, then build a prompt that summarizes the locations for a specific user query (e.g. “find coffee shops nearby”).

Haystack 2.29.0

Tue, 12 May 2026 00:00:00 +0000

⭐️ Highlights

🔍 Combine Retrievers with `MultiRetriever` and `TextEmbeddingRetriever`

Two new retriever components make it easier to build hybrid search pipelines. MultiRetriever runs multiple text retrievers in parallel and merges their results into a single deduplicated list, ranked by reciprocal rank fusion by default. You can selectively enable or disable individual retrievers at runtime using the active_retrievers parameter. This is useful when you want to skip the embedding retriever for short or keyword-only queries, for example.

Use the ⚡ vLLM inference engine with Haystack

Wed, 22 Apr 2026 00:00:00 +0000

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs.

This notebook shows how to use it with Haystack.

Install vLLM + Haystack integration

we install vLLM using uv ( installation docs). For production use cases, there are also other options, including Docker ( docs).
we also install vllm-haystack, the vLLM/Haystack integration.

! uv pip install vllm vllm-haystack nest-asyncio python-weather

Serving and using Generative Language Models

vLLM primarily supports most open-weights Generative Language Models.

vllm serve launches an OpenAI-compatible server.

Build with Gemma 4 and Haystack

Mon, 20 Apr 2026 00:00:00 +0000

Gemma 4 is a family of great open models by Google.

Some interesting facts:

4 sizes (E2B, E4B, 26B4A, 31B)
Reasoning models
Support for tool calling and structured outputs
Multimodal
Apache 2.0 license
Up to 256K context window
Trained on 140+ languages

We will see some examples of what we can build with Gemma 4 and the Haystack LLM framework: from RAG to multimodal agents.

Installation

First of all, to have the model running fast enough, you need to Enable GPU Runtime in Colab.

Context Engineering for Agentic Systems: What Goes Into Your Agent's Mind

Mon, 20 Apr 2026 00:00:00 +0000

Every new generation of Large Language Models arrives with a bigger context window - and the temptation to use it fully. If the model can read a million tokens, why not feed it everything? In practice, more context doesn’t reliably mean better answers: it often means higher costs, slower responses, and a model that loses track of what actually matters. Context engineering is the discipline of deciding not just what to put in the context window, but how much, in what form, and when to leave things out - and it’s quickly becoming one of the most important skills in building reliable agentic systems.

Haystack 2.28.0

Mon, 20 Apr 2026 00:00:00 +0000

⭐️ Highlights

🔗 Passing State to Tools and Components

Tools and components can now access the live agent State directly - no extra wiring needed. Just add a state: State parameter to your tool or component’s run method and ToolInvoker automatically injects the current state at runtime. The State parameter is hidden from the LLM-facing schema so the model is never asked to supply it.

For function-based tools created with @tool:

Tabular Data Processing with Prior Labs MCP

Wed, 15 Apr 2026 00:00:00 +0000

In this recipe, we scrape hotel listings from the web, structure them into a table using an LLM, and then use a Haystack Agent equipped with Prior Labs’ TabPFN tools to predict and fill in missing attributes.

Services used:

Firecrawl — web scraping
Anthropic Claude — LLM for extraction and agent reasoning
Prior Labs MCP — tabular ML predictions via MCP tools

Install dependencies

!pip install -q haystack-ai anthropic-haystack mcp-haystack trafilatura firecrawl-haystack

Set up API keys

You’ll need three API keys:

Haystack 2.27.0

Wed, 01 Apr 2026 00:00:00 +0000

⭐️ Highlights

🔌 Automatic List Joining in Pipeline

When a component expects a list as input, pipelines now automatically join multiple inputs into that list (no extra components needed), even if they come in different but compatible types. This enables patterns like combining a plain query string with a list of ChatMessage objects into a single list[ChatMessage] input.

Supported conversations:

Source Types	Target Type	Behavior
T + T	list[T]	Combines multiple inputs into a list of the same type.
T + list[T]	list[T]	Merges single items and lists into a single list.
str + ChatMessage	list[str]	Converts all inputs to `str` and combines into a list.
str + ChatMessage	list[ChatMessage]	Converts all inputs to `ChatMessage` and combines into a list.

Learn more about how to simplify list joins in pipelines in 📖 Smart Pipeline Connections: Implicit List Joining

Haystack 2.26.0

Wed, 18 Mar 2026 00:00:00 +0000

⭐ Highlights

🧠 More Flexible Agents with Dynamic System Prompts

Agent now supports Jinja2 templating in system_prompt, enabling runtime parameter injection and conditional logic directly in system messages. This makes it easier to adapt agent behavior dynamically (e.g. language, tone, time-aware responses) and reuse agents across contexts without redefining prompts

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[weather_tool],
    system_prompt="""{% message role='system' %}
    You always respond in {{language}}.
    {% endmessage %}""",
    required_variables=["language"],
)

result = agent.run(
    messages=[ChatMessage.from_user("What is the weather in London?")], 
    language="Italian" # required variable for the system prompt
)
print(result["last_message"].text)
# >> Il tempo a Londra è soleggiato.

🔍 `LLMRanker` for LLM-Based Reranking

LLMRanker introduces LLM-powered reranking, treating relevance as a semantic reasoning task rather than similarity scoring. This can yield better results for complex or multi-step queries compared to cross-encoders. The component can also filter out irrelevant/duplicate documents entirely, helping provide higher-quality context in RAG pipelines and agent workflows while keeping context windows lean.

Multimodal Search with Gemini Embedding 2 in Haystack

Tue, 10 Mar 2026 00:00:00 +0000

Embeddings are the backbone of modern AI applications, from semantic search and recommendation systems to Retrieval-Augmented Generation (RAG). However, most embedding models operate in a single modality, typically focusing only on textual data.

Google has introduced Gemini Embedding 2, a fully multimodal embedding model that maps text, images, video, audio, and PDFs into a shared vector space. This means you can search across different types of data using a single embedding model: gemini-embedding-2.

Haystack 2.25.2

Thu, 05 Mar 2026 00:00:00 +0000

🐛 Bug Fixes

Reverts the change that made Agent messages optional as it caused issues with pipeline execution. As a consequence, the LLM component now defaults to an empty messages list unless provided at runtime.

Advanced RAG: Query Decomposition and Reasoning

Wed, 04 Mar 2026 00:00:00 +0000

by Tuana Celik ( LI, Twitter)

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval cookbook & full article

2️⃣ Query Expansion cookbook & full article

3️⃣ Query Decomposition & the full article

4️⃣ Automated Metadata Enrichment

Query decomposition is a technique we can use to decompose complex queries into simpler steps, answering each sub-question, and getting an LLM to reason about the final answer based on the answers to the sub-questions.

Agent-Powered Retrieval with Haystack

Tue, 03 Mar 2026 00:00:00 +0000

Notebook by Bilge Yücel

In this notebook, you’ll build an intelligent movie recommendation assistant powered by Haystack and Qdrant. You’ll learn how to combine sparse vector search, metadata filtering (payload), and LLM-based agents to create a system that can understand natural language queries and recommend relevant movies from a curated dataset.

By the end of this notebook, you’ll have a fully working assistant that can answer queries like:

“Find me a highly-rated action movie about car racing.”
“Can you recommend five japanese thrillers?”
“What can I watch with my kids, about animals?”

This assistant will be implemented as a tool-calling agent and have access to a retrieval_tool that can retrieve the information from the database based on the generated query and filters.

Build with Llama Stack and Haystack Agent

Tue, 03 Mar 2026 00:00:00 +0000

This notebook demonstrates how to use the LlamaStackChatGenerator component with Haystack Agent to enable function calling capabilities. We’ll create a simple weather tool that the Agent can call to provide dynamic, up-to-date information.

We start with installing integration package.

%%bash

pip install llama-stack-haystack

Setup

Before running this example, you need to:

Set up Llama Stack Server through an inference provider
Have a model available (e.g., llama3.2:3b)

For a quick start on how to setup server with Ollama, see the Llama Stack documentation.

DevOps Support Agent with Human in the Loop

Tue, 03 Mar 2026 00:00:00 +0000

Notebook by Amna Mubashar

This notebook demonstrates how a Haystack Agent can interactively ask for user input when it’s uncertain about the next step. We’ll build a Human-in-the-Loop tool that the Agent can call dynamically. When the Agent encounters ambiguity or incomplete information, it will ask for more input from the human to continue solving the task.

For this purpose, we will create a DevOps Support Agent.

CI/CD pipelines occasionally fail for reasons that can be hard to diagnose including manually—broken tests, mis-configured environment variables, flaky integrations, etc.

Multimodal Agent with fastRAG and Haystack

Tue, 03 Mar 2026 00:00:00 +0000

Notebook by Bilge Yucel ( X, Linkedin) and Moshe Berchansky ( Linkedin)

👉 Follow Experimental Multimodal Agent example to build with the native Haystack agent 👈

In this cookbook, we’ll show you how to use the 4B Phi3.5 Vision model to build a multimodal conversational agent. This agent will use tools to perform multi-step reasoning through ReAct prompting and answer questions about the nutrition facts of different foods, leveraging Haystack and fastRAG.

fastRAG is a research framework developed by Intel Labs for efficient and optimized RAG pipelines. It is fully compatible with Haystack and includes novel and efficient RAG modules designed for efficient deployment on Intel hardware, including client and server CPUs (Xeon) and the Intel Gaudi AI accelerator.

Sparse Embedding Retrieval with Qdrant and FastEmbed

Tue, 03 Mar 2026 00:00:00 +0000

In this notebook, we will see how to use Sparse Embedding Retrieval techniques (such as SPLADE) in Haystack.

We will use the Qdrant Document Store and FastEmbed Sparse Embedders.

Why SPLADE?

Sparse Keyword-Based Retrieval (based on BM25 algorithm or similar ones) is simple and fast, requires few resources but relies on lexical matching and struggles to capture semantic meaning.
Dense Embedding-Based Retrieval takes semantics into account but requires considerable computational resources, usually does not work well on novel domains, and does not consider precise wording.

While good results can be achieved by combining the two approaches ( tutorial), SPLADE (Sparse Lexical and Expansion Model for Information Retrieval) introduces a new method that encapsulates the positive aspects of both techniques. In particular, SPLADE uses Language Models like BERT to weigh the relevance of different terms in the query and perform automatic term expansions, reducing the vocabulary mismatch problem (queries and relevant documents often lack term overlap).

🦍 Information Extraction with Gorilla

Tue, 03 Mar 2026 00:00:00 +0000

Notebook by Stefano Fiorucci

In this experiment, we will use Large Language Models to perform Information Extraction from textual data.

🎯 Goal: create an application that, given a text (or URL) and a specific structure provided by the user, extracts information from the source.

The “function calling” capabilities of OpenAI models unlock this task: the user can describe a structure, by defining a fake function with all its typed and specific parameters. The LLM will prepare the data in this specific form and send it back to the user.

Advanced RAG: Automated Structured Metadata Enrichment

Sun, 01 Mar 2026 00:00:00 +0000

by Tuana Celik ( LI, Twitter)

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval cookbook & full article

2️⃣ Query Expansion cookbook & full article

3️⃣ Query Decomposition cookbook & the full article

4️⃣ Automated Metadata Enrichment

In this example, you’ll see how you can make use of structured outputs which is an option for some LLMs, and a custom Haystack component, to automate the enrichment of metadata from documents.

Analyze Your Instagram Comments’ Vibe with Apify and Haystack

Sun, 01 Mar 2026 00:00:00 +0000

Author: Jiri Spilka ( Apify)
Idea: Bilge Yücel ( deepset.ai)

Ever wondered if your Instagram posts are truly vibrating among your audience? In this cookbook, we’ll show you how to use the Instagram Comment Scraper Actor to download comments from your instagram post and analyze them using a large language model. All performed within the Haystack ecosystem using the apify-haystack integration.

We’ll start by using the Actor to download the comments, clean the data with the DocumentCleaner and then use the OpenAIChatGenerator to discover the vibe of the Instagram posts.

Chat With Your SQL Database

Sun, 01 Mar 2026 00:00:00 +0000

by Tuana Celik ( X, LI)

In this example, we are querying a SQL Database!

Resources:

Install dependencies

For this demo, we’re using SQLite.

The first few code cells in this section fetchers a CSV file on ‘Absenteeism’ and creates a SQL table from it

!pip install git+https://github.com/deepset-ai/haystack.git@main#egg=haystack-ai

from urllib.request import urlretrieve
from zipfile import ZipFile
import pandas as pd

url = "https://archive.ics.uci.edu/static/public/445/absenteeism+at+work.zip"

# download the file
urlretrieve(url, "Absenteeism_at_work_AAA.zip")

print("Extracting the Absenteeism at work dataset...")
# Extract the CSV file
with ZipFile("Absenteeism_at_work_AAA.zip", 'r') as zf:
    zf.extractall()

# Check the extracted CSV file name (in this case, it's "Absenteeism_at_work.csv")
csv_file_name = "Absenteeism_at_work.csv"

print("Cleaning up the Absenteeism at work dataset...")
# Data clean up
df = pd.read_csv(csv_file_name, sep=";")
df.columns = df.columns.str.replace(' ', '_')
df.columns = df.columns.str.replace('/', '_')

Extracting the Absenteeism at work dataset...
Cleaning up the Absenteeism at work dataset...

columns = df.columns.to_list()
columns = ', '.join(columns)
columns

'ID, Reason_for_absence, Month_of_absence, Day_of_the_week, Seasons, Transportation_expense, Distance_from_Residence_to_Work, Service_time, Age, Work_load_Average_day_, Hit_target, Disciplinary_failure, Education, Son, Social_drinker, Social_smoker, Pet, Weight, Height, Body_mass_index, Absenteeism_time_in_hours'

import sqlite3

connection = sqlite3.connect('absenteeism.db')
print("Opened database successfully");

connection.execute('''CREATE TABLE IF NOT EXISTS absenteeism (ID integer,
                        Reason_for_absence integer,
                        Month_of_absence integer,
                        Day_of_the_week integer,
                        Seasons integer,
                        Transportation_expense integer,
                        Distance_from_Residence_to_Work integer,
                        Service_time integer,
                        Age integer,
                        Work_load_Average_day_ integer,
                        Hit_target integer,
                        Disciplinary_failure integer,
                        Education integer,
                        Son integer,
                        Social_drinker integer,
                        Social_smoker integer,
                        Pet integer,
                        Weight integer,
                        Height integer,
                        Body_mass_index integer,
                        Absenteeism_time_in_hours integer);''')
connection.commit()

Opened database successfully

df.to_sql('absenteeism', connection, if_exists='replace', index = False)

connection.close()

Create a SQL Query Component

Here, we’re creating a custom component called SQLQuery, this way, we can use it in our Haystack pipeline like any other component (like a retriever, generator etc). This component does just one thing:

RAG Pipeline Evaluation Using RAGAS

Sun, 01 Mar 2026 00:00:00 +0000

Ragas is an open source framework for model-based evaluation to evaluate your Retrieval Augmented Generation (RAG) pipelines and LLM applications. It supports metrics like correctness, tone, hallucination (faithfulness), fluency, and more.

For more information about evaluators, supported metrics and usage, check out:

This notebook shows how to use the Ragas-Haystack integration to evaluate a RAG pipeline against various metrics.

Notebook by Anushree Bannadabhavi, Siddharth Sahu, Julian Risch

Prerequisites:

Ragas uses OpenAI key for computing some metrics, so we need an OpenAI API key.

import os
from getpass import getpass

os.environ["OPENAI_API_KEY"] = getpass("Enter OpenAI API key:")

Install dependencies

!pip install ragas-haystack

Importing Required Libraries

from haystack import Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.embedders import OpenAITextEmbedder, OpenAIDocumentEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import AnswerBuilder
from haystack_integrations.components.evaluators.ragas import RagasEvaluator

from ragas.llms import HaystackLLMWrapper
from ragas.metrics import AnswerRelevancy, ContextPrecision, Faithfulness

Creating a Sample Dataset

In this section we create a sample dataset containing information about AI companies and their language models. This dataset serves as the context for retrieving relevant data during pipeline execution.

RAG: Web Search and Analysis with Apify and Haystack

Sun, 01 Mar 2026 00:00:00 +0000

Want to give any of your LLM applications the power to search and browse the web? In this cookbook, we’ll show you how to use the RAG Web Browser Actor to search Google and extract content from web pages, then analyze the results using a large language model - all within the Haystack ecosystem using the apify-haystack integration.

This cookbook also demonstrates how to leverage the RAG Web Browser Actor with Haystack to create powerful web-aware applications. We’ll explore multiple use cases showing how easy it is to:

Haystack 2.25.1

Fri, 27 Feb 2026 00:00:00 +0000

⚡️ Enhancement Notes

Auto variadic sockets now also support Optional[list[...]] input types, in addition to plain list[...].

🐛 Bug Fixes

Fixed smart connection logic to support connecting multiple outputs to a socket whose type is Optional[list[...]] (e.g. list[ChatMessage] | None). Previously, connecting two list[ChatMessage] outputs to Agent.messages would fail after its type was updated from list[ChatMessage] to list[ChatMessage] | None.

Haystack 2.25.0

Thu, 26 Feb 2026 00:00:00 +0000

⭐️ Highlights

🛠️ Dynamic Tool Discovery with `SearchableToolset`

For applications with large tool catalogs, we’ve added the SearchableToolset. Instead of exposing all tools upfront, agents start with a single search_tools function and dynamically discover relevant tools using BM25-based keyword search.

This is particularly useful when connecting MCP servers via MCPToolset, where many tools may be available. By combining the two, agents can load only the tools they actually need at runtime, reducing context usage and improving tool selection.

Invoking APIs with OpenAPITool

Sun, 22 Feb 2026 00:00:00 +0000

OpenAPITool is discontinued and removed in haystack-experimental==0.4.0. As an alternative, you can use openapi-llm.

Many APIs available on the Web provide an OpenAPI specification that describes their structure and syntax.

OpenAPITool is an experimental Haystack component that allows you to call an API using payloads generated from human instructions.

Here’s a brief overview of how it works:

At initialization, it loads the OpenAPI specification from a URL or a file.
At runtime:
- Converts human instructions into a suitable API payload using a Chat Language Model (LLM).
- Invokes the API.
- Returns the API response, wrapped in a Chat Message.

Let’s see this component in action…

Hacker News Summaries with Custom Components

Thu, 19 Feb 2026 00:00:00 +0000

by Tuana Celik: Twitter, LinkedIn

📚 Check out the Customizing RAG Pipelines to Summarize Latest Hacker News Posts with Haystack article for a detailed run through of this example.

Install dependencies

!pip install newspaper3k
!pip install haystack-ai

Create a Custom Haystack Component

This HackernewsNewestFetcher ferches the last_k newest posts on Hacker News and returns the contents as a List of Haystack Document objects

from typing import List
from haystack import component, Document
from newspaper import Article
import requests

@component
class HackernewsNewestFetcher():

  @component.output_types(articles=List[Document])
  def run(self, last_k: int):
    newest_list = requests.get(url='https://hacker-news.firebaseio.com/v0/newstories.json?print=pretty')
    articles = []
    for id in newest_list.json()[0:last_k]:
      article = requests.get(url=f"https://hacker-news.firebaseio.com/v0/item/{id}.json?print=pretty")
      if 'url' in article.json():
        articles.append(article.json()['url'])

    docs = []
    for url in articles:
      try:
        article = Article(url)
        article.download()
        article.parse()
        docs.append(Document(content=article.text, meta={'title': article.title, 'url': url}))
      except:
        print(f"Couldn't download {url}, skipped")
    return {'articles': docs}

Create a Haystack 2.0 RAG Pipeline

This pipeline uses the components available in the Haystack 2.0 preview package at time of writing (22 September 2023) as well as the custom component we’ve created above.

LinkedIn, Company Intelligence & Lead Enrichment with Haystack, MongoDB Atlas, and Bright Data

Tue, 17 Feb 2026 00:00:00 +0000

🚀 Build Your Own AI Sales Research Assistant

This cookbook demonstrates how to build an AI-powered sales research assistant that:

Extracts live data from LinkedIn, Crunchbase, news sources, and job postings
Stores and indexes data in MongoDB Atlas for semantic search
Answers complex questions like “What pain points is this company facing?” and “Generate a personalized outreach angle”

The Tech Stack:

🌐 Bright Data: Web scraping for 45+ data sources (LinkedIn, Crunchbase, news, job boards)
🍃 MongoDB Atlas: Vector database for semantic search + structured metadata filtering
🔧 Haystack: Open-source LLM framework for building RAG pipelines
🤖 Google Gemini 2.5: Generate actionable sales intelligence from raw data

What You’ll Build:

Haystack 2.24.0

Thu, 12 Feb 2026 00:00:00 +0000

⭐️ Highlights

🔌 Pipelines got simpler

With the updated logic, Pipelines can now:

connect multiple list[T] component outputs directly to a single list[T] input of the next component, simplifying pipeline definitions when multiple components produce compatible outputs. E.g., you can directly connect multiple converters to a writer component without a DocumentJoiner component in ingestion pipelines.
support the automatic conversion between ChatMessage and str types, enabling simpler connections between various components. E.g., you can easily connect an Agent component (which returns ChatMessage as last_message) to a text embedder component (which expects a str as query) without an OutputAdapter component.
automatically convert list[ChatMessage] to ChatMessage and list[str] to str by taking the first element of the list. Another supported conversion is list[ChatMessage] to str, enabling the connection between a chat generator (which returns list[ChatMessage] as messages) and a BM25 retriever (which expects a str as query).
perform list wrapping: a component returning type T can be connected to a component expecting type list[T].

Together, these changes eliminate the need for OutputAdapter and some joiners (ListJoiner, DocumentJoiner) in many common setups such as query rewriting and hybrid search.

Haystack 2.24.1

Thu, 12 Feb 2026 00:00:00 +0000

🐛 Bug Fixes

Fixed a bug in flexible Pipeline connections that prevented automatic value conversion when the receiving component expects a Union type. For example, connecting a component returning ChatMessage to a receiver expecting list[str] | list[ChatMessage] should have worked but did not. The conversion strategy now correctly evaluates each branch of a Union receiver and picks the best match.

Haystack 2.23.0

Tue, 27 Jan 2026 00:00:00 +0000

⭐️ Highlights

🔄 Human-in-the-Loop for Agents

Agents can now pause for human confirmation before executing tools. You can define confirmation behavior per tool: always ask, ask only on first use, or never ask* and fully customize the confirmation UI. This makes it much easier to build safer, more transparent agent workflows, especially when tools trigger side effects or access sensitive data.

agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4.1"),
    tools=[balance_tool, addition_tool, phone_tool],
    system_prompt="You are a helpful financial assistant. Use the provided tool to get bank balances when needed.",
    confirmation_strategies={
        balance_tool.name: BlockingConfirmationStrategy(
            confirmation_policy=AlwaysAskPolicy(), confirmation_ui=RichConsoleUI(console=cons),
        ),
        phone_tool.name: BlockingConfirmationStrategy(
            confirmation_policy=AskOncePolicy(), confirmation_ui=SimpleConsoleUI(),
        ),
        addition_tool.name: BlockingConfirmationStrategy(
            confirmation_policy=NeverAskPolicy(), confirmation_ui=SimpleConsoleUI(),
        )
    },
)

For a detailed walkthrough of confirmation strategies and UI customization, see Tutorial: Human-in-the-Loop with Haystack Agents

Cohere v3 for Multilingual QA

Mon, 26 Jan 2026 00:00:00 +0000

Notebook by Bilge Yucel

Multilingual Generative QA Using Cohere and Haystack

In this notebook, we’ll delve into the details of multilingual retrieval and multilingual generation, and demonstrate how to build a Retrieval Augmented Generation (RAG) pipeline to generate answers from multilingual hotel reviews using Cohere models and Haystack. 🏡

Haystack Useful Sources

For Haystack 1.x version, check out Article: Multilingual Generative Question Answering with Haystack and Cohere

Installation

Let’s start by installing Haystack’s Cohere integration:

!pip install cohere-haystack

Storing Multilingual Embeddings

To create a question answering system for hotel reviews, the first thing we need is a document store. We’ll use an InMemoryDocumentStore to save the hotel reviews along with their embeddings.

PDF-Based Question Answering with Amazon Bedrock and Haystack

Mon, 26 Jan 2026 00:00:00 +0000

Notebook by Bilge Yucel

Amazon Bedrock is a fully managed service that provides high-performing foundation models from leading AI startups and Amazon through a single API. You can choose from various foundation models to find the one best suited for your use case.

In this notebook, we’ll go through the process of creating a generative question answering application tailored for PDF files using the newly added Amazon Bedrock integration with Haystack and OpenSearch to store our documents efficiently. The demo will illustrate the step-by-step development of a QA application designed specifically for the Bedrock documentation, demonstrating the power of Bedrock in the process 🚀

Question Answering with Amazon Sagemaker, Chroma and Haystack

Mon, 26 Jan 2026 00:00:00 +0000

Notebook by Sara Zanzottera and Bilge Yucel

Amazon Sagemaker is a comprehensive, fully managed machine learning service that allows data scientists and developers to build, train, and deploy ML models efficiently. You can choose from various foundation models to find the one best suited for your use case.

In this notebook, we’ll go through the process of creating a generative question answering application using the newly added Amazon Sagemaker integration with Haystack and Chroma to store our documents efficiently. The demo will illustrate the step-by-step development of a QA application using some Wikipedia pages about NASA’s Mars missions 🚀

Domain-Aware UI/UX Reviewer Agent: Custom Tools with Retry and Fallback

Fri, 16 Jan 2026 00:00:00 +0000

Notebook by Mayank Laddha

In this notebook, we’ll build an agent to analyze a web page using both desktop and mobile views. We define simple, optional tools to obtain readability scores, HTML structure, and performance-related data as needed.

Additionally, we’ll extend the Tool class to implement retry and fallback mechanisms for tools.

Our stack:

Haystack Agentic framework - to build our agent
Playwright - for taking screenshots of the website
Beautiful Soup - for scraping HTML text content
Textstat - for calculating text readability

Setting up the Environment

First, we’ll install all the necessary dependencies:

Haystack 2.22.0

Thu, 08 Jan 2026 00:00:00 +0000

⭐️ Highlights

✂️ Smarter Document Chunking with Embedding-Based Splitting

Introducing the new EmbeddingBasedDocumentSplitter, a component that takes an embedder and splits documents based on semantic similarity rather than fixed sizes or rules.

from haystack.components.embedders import SentenceTransformersDocumentEmbedder
from haystack.components.preprocessors import EmbeddingBasedDocumentSplitter

# Initialize an embedder to calculate semantic similarities
embedder = SentenceTransformersDocumentEmbedder()

# Configure the splitter with parameters that control splitting behavior
splitter = EmbeddingBasedDocumentSplitter(
    document_embedder=embedder,
    sentences_per_group=2,      # Group 2 sentences before calculating embeddings
    percentile=0.95,            # Split when cosine distance exceeds 95th percentile
    min_length=50,              # Merge splits shorter than 50 characters
    max_length=1000             # Further split chunks longer than 1000 characters
)
result = splitter.run(documents=[doc])

🔥 `warm_up` Runs Automatically on First Use

Components that define awarm_up method now run it automatically on first execution, removing the need for manual calls and preventing errors in standalone usage.

Haystack Ecosystem: One Name, One Product Family, One Look

Fri, 19 Dec 2025 00:00:00 +0000

We’re making some naming and visual updates at deepset to better reflect the role Haystack already plays as a framework, a community, and the foundation of our enterprise platform.

If you’re already building with Haystack, nothing is changing in how you build or run applications. This update is about clarity, making the Haystack ecosystem easier to understand, easier to navigate, and centered around a single open foundation.

The Open Source to Enterprise Story of Haystack

Haystack began as an open-source framework for building NLP pipelines, created to give developers precise control over how AI systems are composed, debugged, and run in production. From the start, it was designed for real-world use, not just experimentation.

Haystack 2.21.0

Tue, 09 Dec 2025 00:00:00 +0000

⭐️ Highlights

🔍 Smarter, Broader Retrieval with Multi-Query RAG

This release introduces three new components that significantly boost retrieval recall in RAG systems by expanding the user query and retrieving documents across multiple reformulations:

QueryExpander generates semantically similar variations of a user query to broaden search coverage.
MultiQueryTextRetriever runs multiple queries in parallel using a text-based retriever (e.g., BM25) and merges results by score.
MultiQueryEmbeddingRetriever performs the same multi-query retrieval flow using embeddings, enabling richer semantic recall.

Used together, these components create a multi-query retrieval pipeline that improves recall especially when queries are short or ambiguous.

Haystack 2.20.0

Thu, 13 Nov 2025 00:00:00 +0000

⭐️ Highlights

Support for OpenAI’s Responses API

Haystack now integrates the OpenAI’s Responses API through the new OpenAIResponsesChatGenerator and AzureOpenAIResponsesChatGenerator components.

This unlocks several advanced capabilities like:

Retrieving concise summaries of the model’s reasoning process.
Using native OpenAI or MCP tool formats alongside Haystack Tool objects and Toolset instances.

Example with reasoning and a web search tool:

from haystack.components.generators.chat import AzureOpenAIResponsesChatGenerator
from haystack.dataclasses import ChatMessage

# with `OpenAIResponsesChatGenerator`
chat_generator = OpenAIResponsesChatGenerator(
    model="o3-mini",
    generation_kwargs={"summary": "auto", "effort": "low"},
    tools=[{"type": "web_search"}],
)
response = chat_generator.run(messages=[ChatMessage.from_user("What's a positive news story from today?")])

# with `AzureOpenAIResponsesChatGenerator`
chat_generator = AzureOpenAIResponsesChatGenerator(
    azure_endpoint="https://example-resource.azure.openai.com/",
    azure_deployment="gpt-5-mini",
    generation_kwargs={"reasoning": {"effort": "low", "summary": "auto"}},
)
response = chat_generator.run(messages=[ChatMessage.from_user("What's Natural Language Processing?")])

print(response["replies"][0].text)

🚀 New Features

Added the AzureOpenAIResponsesChatGenerator, a new component that integrates Azure OpenAI’s Responses API into Haystack.
Added the OpenAIResponsesChatGenerator, a new component that integrates OpenAI’s Responses API into Haystack.
If logprobs are enabled in the generation kwargs, return logprobs in ChatMessage.meta for OpenAIChatGenerator and OpenAIResponsesChatGenerator.
Added an extra field to ToolCall and ToolCallDelta to store provider-specific information.
Updated serialization and deserialization of PipelineSnapshots to work with pydantic BaseModels.
Added async support to SentenceWindowRetriever with a new run_async() method, allowing the retriever to be used in async pipelines and workflows.
Added warm_up() method to all ChatGenerator components (OpenAIChatGenerator, AzureOpenAIChatGenerator, HuggingFaceAPIChatGenerator, HuggingFaceLocalChatGenerator, and FallbackChatGenerator) to properly initialize tools that require warm-up before pipeline execution. The warm_up() method is idempotent and follows the same pattern used in Agent and ToolInvoker components. This enables proper tool initialization in pipelines that use ChatGenerators with tools but without an Agent component.
The AnswerBuilder component now exposes a new parameter return_only_referenced_documents (default: True) that controls if only documents referenced in the replies are returned. Returned documents include two new fields in the meta dictionary:
- source_index: the 1-based index of the document in the input list
- referenced: a boolean value indicating if the document was referenced in the replies (only present if the reference_pattern parameter is provided). These additions make it easier to display references and other sources within a RAG pipeline.

⚡️ Enhancement Notes

Adds generation_kwargs to the Agent component, allowing for more fine-grained control at run-time over chat generation.
Added a revision parameter to all Sentence Transformers embedder components (SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder, SentenceTransformersSparseDocumentEmbedder, and SentenceTransformersSparseTextEmbedder) to allow users to specify a specific model revision/version from the Hugging Face Hub. This enables pinning to a particular model version for reproducibility and stability.
Updated the components Agent, LLMMetadataExtractor, LLMMessagesRouter, and LLMDocumentContentExtractor to automatically call self.warm_up() at runtime if they have not been warmed up yet. This ensures that the components are ready for use without requiring an explicit warm-up call. This differs from previous behavior where warm-up had to be manually invoked before use, otherwise a RuntimeError was raised.
Improved log-trace correlation for DatadogTracer by using the official ddtrace.tracer.get_log_correlation_context() method.
Improved Toolset warm-up architecture for better encapsulation. The base Toolset.warm_up() method now warms up all tools by default, while subclasses can override it to customize initialization (e.g., setting up shared resources instead of warming individual tools). The warm_up_tools() utility function has been simplified to delegate to Toolset.warm_up().

🐛 Bug Fixes

Fixed deserialization of state schema when it is None in Agent.from_dict.

Improving PostgreSQL Keyword Search to Avoid Empty Results

Mon, 10 Nov 2025 00:00:00 +0000

Notebook by Mayank Laddha

As noted in the Haystack documentation for PgvectorKeywordRetriever, this component, unlike others such as ElasticsearchBM25Retriever, doesn’t apply fuzzy search by default. As a result, queries need to be crafted carefully to avoid returning empty results.

In this notebook, you’ll extend PgvectorDocumentStore to make it more forgiving and flexible. You’ll learn how to subclass it to use PostgreSQL’s websearch_to_tsquery and how to leverage NLTK to extract keywords and transform user queries.

Lufthansa Industry Solutions Uses Haystack to Power Enterprise RAG

Fri, 24 Oct 2025 00:00:00 +0000

When you think of Lufthansa, you might picture planes, airports, or global travel, but Lufthansa Industry Solutions (LHIND) is making an impact in a different way: as a full-service IT company delivering digital solutions for clients both inside and outside the Lufthansa Group.

At LHIND, a subsidiary of the Lufthansa Group, teams work on a wide range of projects that span cloud infrastructure, AI, and enterprise data systems to custom software development, process automation, and digital transformation initiatives. Among them is SmartAssistantAI, an enterprise AI chatbot implementation to make company knowledge accessible to everyone, instantly and securely.

Haystack 2.19.0

Mon, 20 Oct 2025 00:00:00 +0000

⭐️ Highlights

🛡️ Try Multiple LLMs with `FallbackChatGenerator`

Introduced FallbackChatGenerator, a resilient chat generator that runs multiple LLMs sequentially and automatically falls back when one fails. It tries each generator in order until one succeeds, handling errors like timeouts, rate limits, or server issues. Ideal for building robust, production-grade chat systems that stay responsive across providers.

from haystack.dataclasses import ChatMessage
from haystack_integrations.components.generators.google_genai import GoogleGenAIChatGenerator
from haystack_integrations.components.generators.anthropic import AnthropicChatGenerator
from haystack.components.generators.chat.openai import OpenAIChatGenerator
from haystack.components.generators.chat.fallback import FallbackChatGenerator

anthropic_generator = AnthropicChatGenerator(model="claude-sonnet-4-5", timeout=1) # force failure with low timeout
google_generator = GoogleGenAIChatGenerator(model="gemini-2.5-flashy") # force failure with typo in model name
openai_generator = OpenAIChatGenerator(model="gpt-4o-mini") # success

chat_generator = FallbackChatGenerator(chat_generators=[anthropic_generator, google_generator, openai_generator])
response = chat_generator.run(messages=[ChatMessage.from_user("What is the plot twist in Shawshank Redemption?")])

print("Successful ChatGenerator: ", response["meta"]["successful_chat_generator_class"])
print("Response: ", response["replies"][0].text)

Output:

How TAC Built an Agentic Chatbot with Haystack to Transform Trade Promotions Workflows

Mon, 06 Oct 2025 00:00:00 +0000

When a leading company like TELUS Agriculture & Consumer Goods (TAC), with a strong presence in agriculture and consumer goods, turns to AI to streamline complex processes, it’s worth taking a closer look.

TELUS Agriculture & Consumer Goods helps businesses optimize everything from supply chains to retail operations. One of their latest innovations: an agentic chatbot powered by Haystack that simplifies how users interact with their trade promotions platform.

We sat down with the team behind this project to learn how they built it, why they chose Haystack, and what advice they have for other teams looking to implement Retrieval-Augmented Generation (RAG) and agent-based AI solutions in production.

Haystack 2.18.1

Mon, 29 Sep 2025 00:00:00 +0000

⚡️ Enhancement Notes

Added tools to Agent run parameters to enhance the agent’s flexibility. Users can now choose a subset of tools for the agent at runtime by providing a list of tool names, or supply an entirely new set by passing Tool objects or a Toolset.

🐛 Bug Fixes

Fix Agent run_async method to correctly handle async streaming callbacks. This previously triggered errors due to a bug.
Prevent duplication of the last assistant message in the chat history when initializing from an AgentSnapshot.
We were setting response_format to None in OpenAIChatGenerator by default which doesn’t follow the API spec. We now omit the variable if response_format is not passed by the user.

Haystack 2.18.0

Mon, 22 Sep 2025 00:00:00 +0000

⭐️ Highlights

🔁 Pipeline Error Recovery with Snapshots

Pipelines now capture a snapshot of the last successful step when a run fails, including intermediate outputs. This lets you diagnose issues (e.g., failed tool calls), fix them, and resume from the checkpoint instead of restarting the entire run. Currently supported for synchronous Pipeline and Agent (not yet in AsyncPipeline)

The snapshot is part of the exception raised with the PipelineRuntimeError when the pipeline run fails. You need to wrap your pipeline.run() in a try-except block.

Haystack 2.17.1

Wed, 20 Aug 2025 00:00:00 +0000

🐛 Bug Fixes

Fixed the from_dict method of MetadataRouter so the output_type parameter introduced in Haystack 2.17 is now optional when loading from YAML. This ensures compatibility with older Haystack pipelines.
In OpenAIChatGenerator, improved the logic to exclude unsupported custom tool calls. The previous implementation caused compatibility issues with the Mistral Haystack core integration, which extends OpenAIChatGenerator.

Haystack 2.17.0

Tue, 19 Aug 2025 00:00:00 +0000

⭐️ Highlights

🖼️ Image support for several model providers

Following the introduction of image support in Haystack 2.16.0, we’ve expanded this to more model providers in Haystack and Haystack Core integrations.

Now supported: Amazon Bedrock, Anthropic, Azure, Google, Hugging Face API, Meta Llama API, Mistral, Nvidia, Ollama, OpenAI, OpenRouter, STACKIT.

🧩 Extended components

We’ve improved several components to make them more flexible:

MetadataRouter, which is used to route Documents based on metadata, has been extended to also support routing ByteStream objects.
The SentenceWindowRetriever, which retrieves neighboring sentences around relevant Documents to provide full context, is now more flexible. Previously, its source_id_meta_field parameter accepted only a single field containing the ID of the original document. It now also accepts a list of fields, so that only documents matching all of the specified meta fields will be retrieved.

⬆️ Upgrade Notes

MultiFileConverter outputs a new key failed in the result dictionary, which contains a list of files that failed to convert. The documents output is included only if at least one file is successfully converted. Previously, documents could still be present but empty if a file with a supported MIME type was provided but did not actually exist.

Introduction to Multimodal Text Generation

Thu, 14 Aug 2025 00:00:00 +0000

In this notebook, we introduce the features that enable multimodal text generation in Haystack.

We introduced the ImageContent dataclass, which represents the image content of a user ChatMessage.
We developed some image converter components.
The OpenAIChatGenerator was extended to support multimodal messages.
The ChatPromptBuilder was refactored to also work with string templates, making it easier to support multimodal use cases.

In this notebook, we’ll introduce all these features, show an application using textual retrieval + multimodal generation, and a multimodal Agent.

Build Browser Agents with Gemini + Playwright MCP

Wed, 13 Aug 2025 00:00:00 +0000

In many real-world scenarios, websites and applications do not offer APIs for programmatic access.

This is where Browser Agents become especially useful: they can interact with web pages just like a human would, by clicking buttons, filling out forms, scrolling, and extracting content.

In this notebook, we’ll explore how to build Browser Agents that can perform various tasks, mostly focused on information gathering, and even extending to image generation.

🧰 Our stack:

Announcing Haystack Enterprise Starter: Best Practices and Support

Fri, 01 Aug 2025 00:00:00 +0000

💙 Thanks to you and all of our amazing community members, the Haystack open source framework has grown into a thriving developer ecosystem, now used by thousands of organizations to power everything from simple Q&A bots to advanced enterprise agents. As more teams run Haystack in production, one thing has become increasingly clear: building reliable AI systems is hard and scaling them securely is even harder.

We’ve had a front-row seat to these challenges. Across GitHub threads, meetups, community calls, and production deployments, developers have consistently asked for engineering support and hands-on guidance to build for their use case, accelerate deployment, improve observability, and scale infrastructure with confidence. These aren’t just feature requests; they reflect the real-world friction points of teams building AI products that actually ship.

Haystack 2.16.0

Tue, 29 Jul 2025 00:00:00 +0000

⭐️ Highlights

🧠 Agent Breakpoints

This release introduces Agent Breakpoints, a powerful new feature that enhances debugging and observability when working with Haystack Agents. You can pause execution mid-run by inserting breakpoints in the Agent or its tools to inspect internal state and resume execution seamlessly. This brings fine-grained control to agent development and significantly improves traceability during complex interactions.

from haystack.dataclasses.breakpoints import AgentBreakpoint, Breakpoint
from haystack.dataclasses import ChatMessage

chat_generator_breakpoint = Breakpoint(
    component_name="chat_generator", 
    visit_count=0, 
    snapshot_file_path="debug_snapshots"
)
agent_breakpoint = AgentBreakpoint(break_point=chat_generator_breakpoint, agent_name='calculator_agent')

response = agent.run(
    messages=[ChatMessage.from_user("What is 7 * (4 + 2)?")],
    break_point=agent_breakpoint
)

🖼️ Multimodal Pipelines and Agents

You can now blend text and image capabilities across generation, indexing, and retrieval in Haystack.

Building an Interactive Feedback Review Agent with Azure AI Search and Haystack

Tue, 08 Jul 2025 00:00:00 +0000

by Amna Mubashar (Haystack), and Khye Wei (Azure AI Search)

This notebook demonstrates how you can build indexing and querying pipelines using Azure AI Search-Haystack integration. Additionally, you’ll develop an interactive feedback review agent leveraging Haystack Tools.

Install the required dependencies

# Install the required dependencies

%pip install "haystack-ai>=2.13.0"
%pip install "azure-ai-search-haystack
!pip install jq
!pip install nltk=="3.9.1"
!pip install jsonschema
!pip install kagglehub

Loading and Preparing the Dataset

We will use an open dataset consisting of approx. 28000 customer reviews for a clothing store. The dataset is available at Shopper Sentiments.

Chroma Indexing and RAG Examples

Tue, 08 Jul 2025 00:00:00 +0000

Install dependencies

# Install the Chroma integration, Haystack will come as a dependency
!pip install -U chroma-haystack "huggingface_hub>=0.22.0"

Indexing Pipeline: preprocess, split and index documents

In this section, we will index documents into a Chroma DB collection by building a Haystack indexing pipeline. Here, we are indexing documents from the VIM User Manuel into the Haystack ChromaDocumentStore.

We have the .txt files for these pages in the examples folder for the ChromaDocumentStore, so we are using the TextFileToDocument and DocumentWriter components to build this indexing pipeline.

Crawl Website Content for Question Answering with Apify

Tue, 08 Jul 2025 00:00:00 +0000

Author: Jiri Spilka ( Apify)

In this tutorial, we’ll use the apify-haystack integration to call Website Content Crawler and crawl and scrape text content from the Haystack website. Then, we’ll use the OpenAIDocumentEmbedder to compute text embeddings and the InMemoryDocumentStore to store documents in a temporary in-memory database. The last step will be a retrieval augmented generation pipeline to answer users’ questions from the scraped data.

Install dependencies

!pip install -q apify-haystack

Set up the API keys

You need to have an Apify account and obtain APIFY_API_TOKEN.

Newsletter Sending Agent with Tools

Tue, 08 Jul 2025 00:00:00 +0000

🧑‍🍳 Demo by Stefano Fiorucci ( X, LinkedIn) and Tuana Celik( X, LinkedIn)

In this recipe, we will build a newsletter sending agent with 3 tools:

A tool that fetches the top stories from Hacker News
A tool that creates newsletters for a particular audience
A tool that can send emails (with Gmail)

This notebook is updated after Haystack 2.9.0. Experimental features in the old version of this notebook are merged into Haystack core package.

RAG Pipeline Evaluation Using DeepEval

Tue, 08 Jul 2025 00:00:00 +0000

DeepEval is a framework to evaluate Retrieval Augmented Generation (RAG) pipelines. It supports metrics like context relevance, answer correctness, faithfulness, and more.

For more information about evaluators, supported metrics and usage, check out:

This notebook shows how to use DeepEval-Haystack integration to evaluate a RAG pipeline against various metrics.

Prerequisites:

OpenAI key
- DeepEval uses for computing some metrics, so we need an OpenAI key.

import os
from getpass import getpass

os.environ["OPENAI_API_KEY"] = getpass("Enter OpenAI API key:")

Enter OpenAI API key: ········

Install dependencies

!pip install haystack-ai
!pip install "datasets>=2.6.1"
!pip install deepeval-haystack

Create a RAG pipeline

We’ll first need to create a RAG pipeline. Refer to this link for a detailed tutorial on how to create RAG pipelines.

Haystack 2.16.1

Sun, 06 Jul 2025 00:00:00 +0000

🐛 Bug Fixes

Improved validation in the ChatMessage.from_user class method. The method now raises an error if neither text nor content_parts are provided. It does not raise an error if text is an empty string.

Haystack 2.15.2

Fri, 04 Jul 2025 00:00:00 +0000

⚡️ Enhancement Notes

We’ve relaxed the requirements for the ToolCallDelta dataclass (introduced in Haystack 2.15). Previously, creating a ToolCallDelta instance required either the parameters argument or the name to be set. This constraint has now been removed to align more closely with OpenAI’s SDK behavior. The change was necessary as the stricter requirement was causing errors in certain hosted versions of open-source models that adhere to the OpenAI SDK specification.

🐛 Bug Fixes

Fixed a bug in the print_streaming_chunk utility function that prevented ToolCall name from being printed.

Build a GitHub PR Creator Agent

Mon, 30 Jun 2025 00:00:00 +0000

In this recipe, we’ll create an Agent that uses tools from Haystack’s GitHub integration. Given a GitHub issue URL, the agent will not only comment on the issue but it will also fork the repository and open a pull request.

Step-by-step, the agent will:

Fetch and parse the issue description and comments
Identify the relevant directories and files
Determine the next steps for resolution and post them as a comment
Fork the repository and create a new branch
Open a pull request from the newly created branch to the original repository

For this, we’ll use Haystack’s Agent component. It implements a tool-calling functionality with provider-agnostic chat model support. We can use Agent either as a standalone component or within a pipeline.

Haystack 2.15.1

Mon, 30 Jun 2025 00:00:00 +0000

🐛 Bug Fixes

Fix _convert_streaming_chunks_to_chat_message which is used to convert Haystack StreamingChunks into a Haystack ChatMessage. This fixes the scenario where one StreamingChunk contains two ToolCallDetlas in StreamingChunk.tool_calls. With this fix this correctly saves both ToolCallDeltas whereas before they were overwriting each other. This only occurs with some LLM providers like Mistral (and not OpenAI) due to how the provider returns tool calls.

Haystack 2.15.0

Thu, 26 Jun 2025 00:00:00 +0000

Release Notes

⭐️ Highlights

Parallel Tool Calling for Faster Agents

ToolInvoker now processes all tool calls passed to run or run_async in parallel using an internal ThreadPoolExecutor. This improves performance by reducing the time spent on sequential tool invocations.
This parallel execution capability enables ToolInvoker to batch and process multiple tool calls concurrently, allowing Agents to run complex pipelines efficiently with decreased latency.
You no longer need to pass an async_executor. ToolInvoker manages its own executor, configurable via the max_workers parameter in init.

Introducing LLMMessagesRouter

The new LLMMessagesRouter component that classifies and routes incoming ChatMessage objects to different connections using a generative LLM. This component can be used with general-purpose LLMs and with specialized LLMs for moderation like Llama Guard.

Haystack 2.14.3

Thu, 19 Jun 2025 00:00:00 +0000

🐛 Bug Fixes

In ConditionalRouter, fixed the to_dict and from_dict methods to properly handle the case when output_type is a List of types or a List of strings. This occurs when a user specifies a route in ConditionalRouter to have multiple outputs.
Fix the serialization of ComponentTool and Tool when specifying outputs_to_string. Previously an error occurred on deserialization right after serializing if outputs_to_string is not None.

Trace and Evaluate RAG with Arize Phoenix

Fri, 13 Jun 2025 00:00:00 +0000

Phoenix is a tool for tracing and evaluating LLM applications. In this tutorial, we will trace and evaluate a Haystack RAG pipeline. We’ll evaluate using three different types of evaluations:

Relevance: Whether the retrieved documents are relevant to the question.
Q&A Correctness: Whether the answer to the question is correct.
Hallucination: Whether the answer contains hallucinations.

ℹ️ This notebook requires an OpenAI API key.

!pip install -q openinference-instrumentation-haystack haystack-ai arize-phoenix

Set API Keys

from getpass import getpass
import os

if not (openai_api_key := os.getenv("OPENAI_API_KEY")):
    openai_api_key = getpass("🔑 Enter your OpenAI API key: ")

os.environ["OPENAI_API_KEY"] = openai_api_key

🔑 Enter your OpenAI API key: ··········

Launch Phoenix and Enable Haystack Tracing

If you don’t have a Phoenix API key, you can get one for free at phoenix.arize.com. Arize Phoenix also provides self-hosting options if you’d prefer to run the application yourself instead.

Summarize Hacker News Posts with Haystack & OPEA

Tue, 10 Jun 2025 00:00:00 +0000

Welcome to this step-by-step tutorial where we’ll build a simple Retrieval-Augmented Generation (RAG) pipeline using Haystack and OPEA. We’ll fetch the newest Hacker News posts, feed them to a lightweight LLM endpoint (OPEAGenerator), and generate concise one-sentence summaries (based on this notebook). Let’s dive in! 🎉

1. Introduction & Motivation

In modern GenAI applications, having a flexible, performant, and scalable platform is essential. OPEA (Open Platform for Enterprise AI) is an open, model-agnostic framework for building and operating composable GenAI solutions. It provides:

Build a GitHub Issue Resolver Agent

Mon, 09 Jun 2025 00:00:00 +0000

In this recipe, we’ll create a GitHub Issue Resolver Agent with Anthropic Claude 4 Sonnet. Given an issue URL, the agent will:

Fetch and parse the issue description and comments
Identify the relevant repository, directories, and files
Retrieve and process file content
Determine the next steps for resolution and post them as a comment

For this, we’ll use the new Agent component. Agent is a Haystack component that implements a tool-calling functionality with provider-agnostic chat model support. We can use Agent either as a standalone component or within a pipeline.

Haystack 2.14.2

Wed, 04 Jun 2025 00:00:00 +0000

🐛 Bug Fixes

Fixed a bug in OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder where if an OpenAI API error occurred mid-batch then the following embeddings would be paired with the wrong documents.

🚀 New Features

Added a raise_on_failure boolean parameter to OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder. If set to True then the component will raise an exception when there is an error with the API request. It is set to False by default so the previous behavior of logging an exception and continuing is still the default.

Haystack 2.14.1

Fri, 30 May 2025 00:00:00 +0000

🐛 Bug Fixes

Fixed a mypy issue in the OpenAIChatGenerator and its handling of stream responses. This issue only occurs with mypy>=1.16.0.
Fix type comparison in schema validation by replacing is not with != when checking the type List[ChatMessage]. This prevents false mismatches due to Python’s is operator comparing object identity instead of equality.

Haystack 2.14.0

Mon, 26 May 2025 00:00:00 +0000

⭐️ Highlights

Enhancements for Complex Agentic Systems

We’ve improved agent workflows with better message handling and streaming support. Agent component now returns a last_message output for quick access to the final message, and can use a streaming_callback to emit tool results in real time. You can use the updated print_streaming_chunk or write your own callback function to enable ToolCall details during streaming.

from haystack.components.websearch import SerperDevWebSearch
from haystack.components.agents import Agent
from haystack.components.generators.utils import print_streaming_chunk
from haystack.tools import tool, ComponentTool
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

web_search = ComponentTool(name="web_search", component=SerperDevWebSearch(top_k=5))
wiki_search = ComponentTool(name="wiki_search", component=SerperDevWebSearch(top_k=5, allowed_domains=["https://www.wikipedia.org/"]))

research_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="""
    You are a research agent that can find information on web or specifically on wikipedia. 
    Use wiki_search tool if you need facts and use web_search tool for latest news on topics.
    Use one tool at a time, use the other tool if the retrieved information is not enough.
    Summarize the retrieved information before returning response to the user.
    """,
    tools=[web_search, wiki_search],
    streaming_callback=print_streaming_chunk
)

result = research_agent.run(messages=[ChatMessage.from_user("Can you tell me about Florence Nightingale's life?")])

Enabling streaming with print_streaming_chunk function looks like this:

Create a Swarm of Agents

Mon, 12 May 2025 00:00:00 +0000

As of Haystack 2.9.0, experimental dataclasses (refactored ChatMessage and ChatRole, ToolCall and Tool) and components (refactored OpenAIChatGenerator, ToolInvoker) are removed from the haystack-experimental and merged into Haystack core.

OpenAI recently released Swarm: an educational framework that proposes lightweight techniques for creating and orchestrating multi-agent systems.

In this notebook, we’ll explore the core concepts of Swarm ( Routines and Handoffs) and implement them using Haystack and its tool support.

This exploration is not only educational: we will unlock features missing in the original implementation, like the ability of using models from various providers. In fact, our final example will include 3 agents: one powered by gpt-4o-mini (OpenAI), one using Claude 3.5 Sonnet (Anthropic) and a third running Llama-3.2-3B locally via Ollama.

Deploy AI Pipelines Faster with Hayhooks

Mon, 12 May 2025 00:00:00 +0000

Haystack is an AI orchestration framework that enables developers to effortlessly build custom AI pipelines using a modular, building-block approach. However, when it’s time to take those pipelines from your development environment to production, you’re often left with a tough decision: write custom server code, or rely on proprietary tools that may not offer the flexibility you need.

We’re excited to announce Hayhooks, an open source package designed to simplify deployment. It lets you focus on developing meaningful AI systems rather than worrying about the underlying infrastructure.

Haystack 2.13.2

Fri, 09 May 2025 00:00:00 +0000

⚡️ Enhancement Notes

Updated pipeline execution logic to use a new utility method _deepcopy_with_exceptions, which attempts to deep copy an object and safely falls back to the original object if copying fails. Additionally _deepcopy_with_exceptions skips deep-copying of Component, Tool, and Toolset instances when used as runtime parameters. This prevents errors and unintended behavior caused by trying to deepcopy objects that contain non-copyable attributes (e.g. Jinja2 templates, clients). Previously, standard deepcopy was used on inputs and outputs which occasionally lead to errors since certain Python objects cannot be deepcopied.

🐛 Bug Fixes

Make internal tool conversion in the HuggingFaceAPICompatibleChatGenerator compatible with huggingface_hub>=0.31.0. In the huggingface_hub library, arguments attribute of ChatCompletionInputFunctionDefinition has been renamed to parameters. Our implementation is compatible with both the legacy version and the new one.
The HuggingFaceAPIChatGenerator now checks the type of the arguments variable in the tool calls returned by the Hugging Face API. If arguments is a JSON string, it is parsed into a dictionary. Previously, the arguments type was not checked, which sometimes led to failures later in the tool workflow.

Simple Keyword Extraction using OpenAIChatGenerator

Fri, 09 May 2025 00:00:00 +0000

This notebook demonstrates how to extract keywords and key phrases from text using Haystack’s ChatPromptBuilder together with an LLM via OpenAIChatGenerator. We will:

Define a prompt that instructs the model to identify single- and multi-word keywords.
Capture each keyword’s character offsets.
Assign a relevance score (0–1).
Parse and display the results as JSON.

Install packages and setup OpenAI API key

!pip install haystack-ai

import os
from getpass import getpass

if "OPENAI_API_KEY" not in os.environ:
    os.environ["OPENAI_API_KEY"] = getpass("Enter OpenAI API key:")

Import Required Libraries

import json


from haystack.dataclasses import ChatMessage
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator

Prepare Text

Collect your text you want to analyze.

Haystack 2.13.1

Thu, 24 Apr 2025 00:00:00 +0000

🐛 Bug Fixes

Update the __deepcopy__ of ComponentTool to gracefully handle NotImplementedError when trying to deepcopy attributes.
Fix an issue where OpenAIChatGenerator and OpenAIGenerator were not properly handling wrapped streaming responses from tools like Weave.
Move deserialize_tools_inplace back to original import path of from haystack.tools.tool import deserialize_tools_inplace.

Haystack 2.13.0

Tue, 22 Apr 2025 00:00:00 +0000

⭐️ Highlights

Enhanced Agent Tracing and Async Support

Haystack’s Agent got several improvements!

Agent Tracing
Agent tracing now provides deeper visibility into the agent’s execution. For every call, the inputs and outputs of the ChatGenerator and ToolInvoker are captured and logged using dedicated child spans. This makes it easier to debug, monitor, and analyze how an agent operates step-by-step.

Below is an example of what the trace looks like in Langfuse:

Haystack RAG Pipeline with Self-Deployed AI models using NVIDIA NIMs

Thu, 17 Apr 2025 00:00:00 +0000

In this notebook, we will build a Haystack Retrieval Augmented Generation (RAG) Pipeline using self-hosted AI models with NVIDIA Inference Microservices or NIMs.

The notebook is associated with a technical blog demonstrating the steps to deploy NVIDIA NIMs with Haystack into production.

The code examples expect the LLM Generator and Retrieval Embedding AI models already deployed using NIMs microservices following the procedure described in the technical blog.

You can also substitute the calls to NVIDIA NIMs with the same AI models hosted by NVIDIA on ai.nvidia.com.

Haystack 2.12.2

Mon, 14 Apr 2025 00:00:00 +0000

🐛 Bug Fixes

Fix ChatMessage.from_dict to handle cases where optional fields like name and meta are missing.
Make Document’s first-level fields to take precedence over meta fields when flattening the dictionary representation.

Haystack 2.12.1

Thu, 10 Apr 2025 00:00:00 +0000

🐛 Bug Fixes

In Agent we make sure state_schema is always initialized to have ‘messages’. Previously this was only happening at run time which is why pipeline.connect failed because output types are set at init time. Now the Agent correctly sets everything in state_schema (including messages by default) at init time.
In AsyncPipline the span tag name is updated from hasytack.component.outputs to haystack.component.output. This matches the tag name used in Pipeline and is the tag name expected by our tracers.

Haystack 2.12.0

Wed, 02 Apr 2025 00:00:00 +0000

⭐️ Highlights

Agent Component with State Management

The Agent component enables tool-calling functionality with provider-agnostic chat model support and can be used as a standalone component or within a pipeline. With SERPERDEV_API_KEY and OPENAI_API_KEY defined, a Web Search Agent is as simple as:

from haystack.components.agents import Agent 
from haystack.components.generators.chat import OpenAIChatGenerator 
from haystack.components.websearch import SerperDevWebSearch 
from haystack.dataclasses import ChatMessage 
from haystack.tools.component_tool import ComponentTool 

web_tool = ComponentTool(     
    component=SerperDevWebSearch(), 
) 

agent = Agent(     
    chat_generator=OpenAIChatGenerator(),
    tools=[web_tool],
) 

result = agent.run(
    messages=[ChatMessage.from_user("Find information about Haystack by deepset")]
)

The Agent supports streaming responses, customizable exit conditions, and a flexible state management system that enables tools to share and modify data during execution:

Improving Retrieval with Auto-Merging and Hierarchical Document Retrieval

Thu, 20 Mar 2025 00:00:00 +0000

This notebook shows how to use Haystack components: AutoMergingRetriever and HierarchicalDocumentSplitter.

📚 Read the full article here

Setting up

!pip install haystack-ai

Let’s get a dataset to index and explore

We will use a dataset containing 2225 new articles part of the paper by “Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering”, Proc. ICML 2006. by D. Greene and P. Cunningham.
The original dataset is available at http://mlg.ucd.ie/datasets/bbc.html, but we will instead use a CSV processed version available here: https://raw.githubusercontent.com/amankharwal/Website-data/master/bbc-news-data.csv

Optimize RAG Applications with Document Reranking Using Haystack With NVIDIA NeMo Retriever

Thu, 20 Mar 2025 00:00:00 +0000

In retrieval-augmented generation (RAG) applications, the quality of the retrieved documents plays a critical role in delivering accurate and meaningful responses. But what happens when embedding similarity is not enough to get an accurate ordering of the reference documents? This is where reranking comes into play.

What’s Reranking?

Reranking refers to assigning a relevance score to each document based on how well it matches the query. Reranking reorders the retrieved documents to ensure the most contextually relevant results are at the top. This is important because while the retrieval stage focuses on recall, considering relevance broadly, reranking “fine-tunes” the results for increased precision.

Haystack 2.11.2

Tue, 18 Mar 2025 00:00:00 +0000

⚡️ Enhancement Notes

Refactored the processing of streaming chunks from OpenAI to simplify logic.
Added tests to ensure expected behavior when handling streaming chunks when using include_usage=True.

🐛 Bug Fixes

Fixed issue with MistralChatGenerator not returning a finish_reason when using streaming. Fixed by adjusting how we look for the finish_reason when processing streaming chunks. Now, the last non-None finish_reason is used to handle differences between OpenAI and Mistral.

Haystack 2.11.1

Thu, 13 Mar 2025 00:00:00 +0000

🐛 Bug Fixes

Add dataframe to legacy fields for the Document dataclass. This fixes a bug where Document.from_dict() in haystack-ai>=2.11.0 could not properly deserialize a Document dictionary obtained with document.to_dict(flatten=False) in haystack-ai<=2.10.0.

Haystack 2.11.0

Mon, 10 Mar 2025 00:00:00 +0000

⭐️ Highlights

Faster Imports

With lazy importing, importing individual components now requires 50% less CPU time on average. Overall import performance has also significantly improved: for example, import haystack now consumes only 2-5% of the CPU time it previously did.

Extended Async Run Support

As of this release, all chat generators and retrievers in the core package now include a run_async method, enabling asynchronous execution at the component level. When used in an AsyncPipeline, this method runs automatically, providing native async capabilities.

Haystack 2.10.3

Thu, 20 Feb 2025 00:00:00 +0000

🐛 Bug Fixes

Fixed accumulation of a tools arguments when streaming with an OpenAIChatGenerator.

Haystack 2.10.2

Wed, 19 Feb 2025 00:00:00 +0000

🐛 Bug Fixes

Pipelines with components that return plain pandas dataframes failed. The comparison of socket values is now ‘is not’ instead of ‘!=’ to avoid errors with dataframes.

Haystack 2.10.1

Tue, 18 Feb 2025 00:00:00 +0000

🐛 Bug Fixes

ComponentTool does not truncate description anymore.

Haystack 2.10.0

Wed, 12 Feb 2025 00:00:00 +0000

⭐️ Highlights

Improved `Pipeline.run()` Logic

The new Pipeline.run() logic fixes common pipeline issues, including exceptions, incorrect component execution, missing intermediate outputs, and premature execution of lazy variadic components. While most pipelines should remain unaffected, we recommend carefully reviewing your pipeline executions if you are using cyclic pipelines or pipelines with lazy variadic components to ensure their behavior has not changed. You can use this tool to compare the execution traces of your pipeline with the old and new logic.

Function Calling with OpenAIChatGenerator

Thu, 30 Jan 2025 00:00:00 +0000

⚠️ As of Haystack 2.9.0, this recipe has been deprecated. For the same example, follow Tutorial: Building a Chat Agent with Function Calling

Notebook by Bilge Yucel ( LI & X (Twitter))

A guide to understand function calling and how to use OpenAI function calling feature with Haystack.

📚 Useful Sources:

Overview

Here are some use cases of function calling from OpenAI Docs:

Create assistants that answer questions by calling external APIs (e.g. like ChatGPT Plugins) e.g. define functions like send_email(to: string, body: string), or get_current_weather(location: string, unit: ‘celsius’ | ‘fahrenheit’)
Convert natural language into API calls e.g. convert “Who are my top customers?” to get_customers(min_revenue: int, created_before: string, limit: int) and call your internal API
Extract structured data from text e.g. define a function called extract_data(name: string, birthday: string), or sql_query(query: string)

Set up the Development Environment

%%bash

pip install haystack-ai==2.8.1

import os
from getpass import getpass
from google.colab import userdata

os.environ["OPENAI_API_KEY"] = userdata.get('OPENAI_API_KEY') or getpass("OPENAI_API_KEY: ")

Learn about the OpenAIChatGenerator

OpenAIChatGenerator is a component that supports the function calling feature of OpenAI.

Use DeepSeek-R1 with Haystack: Demo and Tutorial

Wed, 29 Jan 2025 00:00:00 +0000

The latest release from DeepSeek confirms an essential truth about AI: there won’t be one model or provider to rule them all. As the field evolves, it’s evident that diverse models bring unique strengths, making a model-agnostic approach vital for developers and organizations alike. Whether you’re building agentic systems, Retrieval-Augmented Generation (RAG) architectures, search or other architectures, a model-agnostic design unlocks flexibility, scalability, and long-term success 🔐

Stay Flexible with a Model-Agnostic Approach

Decoupling your application from specific models or APIs gives you the freedom to adapt as AI evolves. A model-agnostic approach lets you choose the best tool for the job—whether it’s generating human-like text, answering complex questions, or handling domain-specific analysis. Through Haystack’s modular architecture, you can easily test, swap, or integrate new models as they emerge, all without rearchitecting your entire AI system. This flexibility ensures you stay ahead of advancements, fine-tune for industry needs, and maintain optimal performance without being locked into a single provider ecosystem.

Build an Agentic RAG Pipeline in deepset Studio

Tue, 14 Jan 2025 00:00:00 +0000

In this article, we’ll explore how to build an agentic Retrieval Augmented Generation (RAG) pipeline on deepset Studio, the AI application prototyping tool for developers. We’ll first build a basic RAG pipeline and then extend the pipeline with a fallback mechanism that can perform a web search if the answer to the user query cannot be found in the database.

This article also serves as a solution to Day 5 challenge of Advent of Haystack 2024: Elves’ Secret for Faster Development 💨

Haystack 2.9.0

Tue, 14 Jan 2025 00:00:00 +0000

⭐️ Highlights

Tool Calling Support

We are introducing the Tool, a simple and unified abstraction for representing tools in Haystack, and the ToolInvoker, which executes tool calls prepared by LLMs. These features make it easy to integrate tool calling into your Haystack pipelines, enabling seamless interaction with tools when used with components like OpenAIChatGenerator and HuggingFaceAPIChatGenerator. Here’s how you can use them:

def dummy_weather_function(city: str):
    return f"The weather in {city} is 20 degrees."

tool = Tool(
    name="weather_tool",
    description="A tool to get the weather",
    function=dummy_weather_function,
    parameters={
      "type": "object",
      "properties": {"city": {"type": "string"}},
      "required": ["city"],
    }
)

pipeline = Pipeline()
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini", tools=[tool]))
pipeline.add_component("tool_invoker", ToolInvoker(tools=[tool]))
pipeline.connect("llm.replies", "tool_invoker.messages")

message = ChatMessage.from_user("How is the weather in Berlin today?")
result = pipeline.run({"llm": {"messages": [message]}})

Use Components as Tools As an abstraction of Tool, ComponentTool allows LLMs to interact directly with components like web search, document processing, or custom user components. It simplifies schema generation and type conversion, making it easy to expose complex component functionality to LLMs.

Haystack 2.8.1

Fri, 10 Jan 2025 00:00:00 +0000

Release Notes

🐛 Bug Fixes

Pin OpenAI client to >=1.56.1 to avoid issues related to changes in the httpx library.
PyPDFToDocument now creates documents with id based on converted text and meta data. Before it didn’t take the meta data into account.
Fixes issues with deserialization of components in multi-threaded environments.

🧩 Quizzes and Adventures 🏰 with Character Codex and llamafile

Tue, 10 Dec 2024 00:00:00 +0000

Let’s build something fun with Character Codex, a newly released dataset featuring popular characters from a wide array of media types and genres…

We’ll be using Haystack for orchestration and llamafile to run our models locally.

We will first build a simple quiz game, in which the user is asked to guess the character based on some clues. Then we will try to get two characters to interact in a chat and maybe even have an adventure together!

Haystack 2.8.0

Thu, 05 Dec 2024 00:00:00 +0000

Release Notes

⬆️ Upgrade Notes

Remove is_greedy deprecated argument from @component decorator. Change the Variadic input of your Component to GreedyVariadic instead.

🚀 New Features

We’ve added a new DALLEImageGenerator component, bringing image generation with OpenAI’s DALL-E to the Haystack

Easy to Use: Just a few lines of code to get started:

from haystack.components.generators import DALLEImageGenerator 

image_generator = DALLEImageGenerator() 
response = image_generator.run("Show me a picture of a black cat.") 
print(response)

Add warning logs to the PDFMinerToDocument and PyPDFToDocument to indicate when a processed PDF file has no content. This can happen if the PDF file is a scanned image. Also added an explicit check and warning message to the DocumentSplitter that warns the user that empty Documents are skipped. This behavior was already occurring, but now its clearer through logs that this is happening.
We have added a new MetaFieldGroupingRanker component that reorders documents by grouping them based on metadata keys. This can be useful for pre-processing Documents before feeding them to an LLM.
Added a new store_full_path parameter to the __init__ methods of the following converters: JSONConverter, CSVToDocument, DOCXToDocument, HTMLToDocument MarkdownToDocument, PDFMinerToDocument, PPTXToDocument, TikaDocumentConverter, PyPDFToDocument , AzureOCRDocumentConverter and TextFileToDocument. The default value is True, which stores full file path in the metadata of the output documents. When set to False, only the file name is stored.
When making function calls via OpenAPI, allow both switching SSL verification off and specifying a certificate authority to use for it.
Add TTFT (Time-to-First-Token) support for OpenAI generators. This captures the time taken to generate the first token from the model and can be used to analyze the latency of the application.
Added a new option to the required_variables parameter to the PromptBuilder and ChatPromptBuilder. By passing required_variables="*" you can automatically set all variables in the prompt to be required.

⚡️ Enhancement Notes

Across Haystack codebase, we have replaced the use of ChatMessage data class constructor with specific class methods (ChatMessage.from_user, ChatMessage.from_assistant, etc.).
Added the Maximum Margin Relevance (MMR) strategy to the SentenceTransformersDiversityRanker. MMR scores are calculated for each document based on their relevance to the query and diversity from already selected documents.
Introduces optional parameters in the ConditionalRouter component, enabling default/fallback routing behavior when certain inputs are not provided at runtime. This enhancement allows for more flexible pipeline configurations with graceful handling of missing parameters.
Added split by line to DocumentSplitter, which will split the document at n.
Change OpenAIDocumentEmbedder to keep running if a batch fails embedding. Now OpenAI returns an error we log that error and keep processing following batches.
Added new initialization parameters to the PyPDFToDocument component to customize the text extraction process from PDF files.
Replace usage of ChatMessage.content with ChatMessage.text across the codebase. This is done in preparation for the removal of content in Haystack 2.9.0.

⚠️ Deprecation Notes

The default value of the store_full_path parameter in converters will change to False in Haysatck 2.9.0 to enhance privacy.
In Haystack 2.9.0, the ChatMessage data class will be refactored to make it more flexible and future-proof. As part of this change, the content attribute will be removed. A new text property has been introduced to provide access to the textual value of the ChatMessage. To ensure a smooth transition, start using the text property now in place of content.
The converter parameter in the PyPDFToDocument component is deprecated and will be removed in Haystack 2.9.0. For in-depth customization of the conversion process, consider implementing a custom component. Additional high-level customization options will be added in the future.
The output of context_documents in SentenceWindowRetriever will change in the next release. Instead of a List[List[Document]], the output will be a List[Document], where the documents are ordered by split_idx_start.

🐛 Bug Fixes

Fix DocumentCleaner not preserving all Document fields when run

Announcing Advent of Haystack 2024 🎄

Mon, 02 Dec 2024 00:00:00 +0000

December is here, and so is the 2nd edition of the Advent of Haystack! This holiday season, we’re inviting the Haystack community to take part in our series of challenges around Generative AI use cases. From mastering the basics of Haystack to building advanced pipelines and intelligent agents with LLMs, this year’s Advent of Haystack is packed with interesting tasks. Whether you’re new to Haystack or a Gen AI pro, there’s something for everyone!

Create a Swarm of Agents

Tue, 26 Nov 2024 00:00:00 +0000

When building applications with Language Models, new patterns are emerging to bridge the gap between the statistical nature of these models and the deterministic logic of traditional programming. Haystack, as an AI framework, supports developers by providing abstractions that simplify this integration.

One of the most promising advances in this space is Tool/function calling, allowing a model to prepare calls for functions. We are working to standardize this capability across different model providers.

Announcing Studio: Your Development Environment for Haystack

Wed, 20 Nov 2024 00:00:00 +0000

We’re thrilled to announce an exciting new addition to the Haystack ecosystem: deepset Studio! After countless requests from our community for a visual editor to create AI workflows and invaluable feedback during the beta phase, we’re officially launching deepset Studio, a powerful tool for visually building, deploying, and managing Haystack pipelines.

What is deepset Studio?

deepset Studio is the community version of deepset Cloud, the enterprise offering from the creators of Haystack. It allows users to visually construct and deploy Haystack pipelines for free. With an intuitive drag-and-drop interface, Studio simplifies the process of designing AI applications by combining Haystack’s core and core-integration components.

Haystack 2.7.0

Mon, 11 Nov 2024 00:00:00 +0000

Release Notes

✨ Highlights

🚅 Rework `Pipeline.run()` logic to better handle cycles

Pipeline.run() internal logic has been heavily reworked to be more robust and reliable than before. This new implementation makes it easier to run Pipelines that have cycles in their graph. It also fixes some corner cases in Pipelines that don’t have any cycle.

📝 Introduce `LoggingTracer`

With the new LoggingTracer, users can inspect the logs in real-time to see everything that is happening in their Pipelines. This feature aims to improve the user experience during experimentation and prototyping.

Building a Multimodal Nutrition Agent

Thu, 07 Nov 2024 00:00:00 +0000

In artificial intelligence, multimodal agents are becoming increasingly popular for their ability to understand and integrate multiple types of input, such as text and images. In this article, we’ll show you how to build a multimodal agent that can interpret both text and image data, like nutrition fact labels on food items, to answer practical questions such as “How much protein is in yogurt?”

We’ll focus on building an agent using Haystack and fastRAG, which can perform multi-step reasoning to extract and provide accurate answers about the nutritional content of different foods.

Design Haystack AI Applications Visually in deepset Studio with NVIDIA NIM

Fri, 01 Nov 2024 00:00:00 +0000

In our previous article, we explored how to build and deploy two key AI pipelines in a retrieval-augmented generation (RAG) application using Haystack with NVIDIA NIM:

Indexing pipeline: Prepares data by preprocessing, chunking, and embedding PDF files, finally storing them in a vector database.
RAG pipeline: Designed to answer questions based on the contents of the uploaded PDF files.

In this article, we’ll take a step further by showing you how to visually design the architecture of these AI pipelines using deepset Studio, a newly released tool to visually create, deploy and test pipelines. With this tool, you can map out the entire structure of your AI workflows, from data ingestion to retrieval, while seamlessly integrating generative AI models accelerated by NVIDIA NIM microservices available in the NVIDIA API catalog.

Haystack 2.6.1

Wed, 16 Oct 2024 00:00:00 +0000

Release Notes

Bug Fixes

Revert change to PyPDFConverter that broke the deserialization of pre 2.6.0 YAMLs.

Haystack 2.6.0

Wed, 02 Oct 2024 00:00:00 +0000

Release Notes

⬆️ Upgrade Notes

gpt-3.5-turbo was replaced by gpt-4o-mini as the default model for all components relying on OpenAI API
Support for the legacy filter syntax and operators (e.g., “$and”, “$or”, “$eq”, “$lt”, etc.), which originated in Haystack v1, has been fully removed. Users must now use only the new filter syntax. See the docs for more details.

🚀 New Features

Added a new component DocumentNDCGEvaluator, which is similar to DocumentMRREvaluator and useful for retrieval evaluation. It calculates the normalized discounted cumulative gain, an evaluation metric useful when there are multiple ground truth relevant documents and the order in which they are retrieved is important.

Advanced RAG: Query Decomposition & Reasoning

Mon, 30 Sep 2024 00:00:00 +0000

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval

2️⃣ Query Expansion

3️⃣ Query Decomposition

4️⃣ Automated Metadata Enrichment

Sometimes a single question is multiple questions in disguise. For example: “Did Microsoft or Google make more money last year?”. To get to the correct answer for this seemingly simple question, we actually have to break it down: “How much money did Google make last year?” and “How much money did Microsoft make last year?”. Only if we know the answer to these 2 questions can we reason about the final answer.

Getting a Daily Digest From Tech Websites

Tue, 24 Sep 2024 00:00:00 +0000

Motivation: We want to stay informed on the latest news in tech. However, with so many websites and news happening every day, it is impossible to keep track of what is going on. But what if we could summarize the latest developments and have all this run locally with an off-the-shelf LLM in a few lines of code?

Let us see how Haystack together with TitanML’s Takeoff Inference Server can help us achieve this.

Hybrid Retrieval with BM42

Tue, 24 Sep 2024 00:00:00 +0000

In this notebook, we will see how to create Hybrid Retrieval pipelines, combining BM42 (a new Sparse embedding Retrieval approach) and Dense embedding Retrieval.

We will use the Qdrant Document Store and Fastembed Embedders.

⚠️ Recent evaluations have raised questions about the validity of BM42. Future developments may address these concerns. Please keep this in mind while reviewing the content.

Why BM42?

Qdrant introduced BM42, an algorithm designed to replace BM25 in hybrid RAG pipelines (dense + sparse retrieval).

🐦‍⬛ Information Extraction with Raven

Thu, 19 Sep 2024 00:00:00 +0000

Notebook by Stefano Fiorucci

In this experiment, we will use Large Language Models to perform Information Extraction from textual data.

🎯 Goal: create an application that, given a URL and a specific structure provided by the user, extracts information from the source.

The “function calling” capabilities of OpenAI models unlock this task: the user can describe a structure, by defining a mock up function with all its typed and specific parameters. The LLM will prepare the data in this specific form and send it back to the user.

Improving Retrieval with Auto-Merging

Thu, 12 Sep 2024 00:00:00 +0000

For most RAG applications, where we first have to retrieve the most relevant context, we end up having to split up documents first, and index those smaller splits of documents. Reasons for this range from needing to retrieve only relevant sections of larger bits of documents to the simple fact that (although they’re improving massively) LLMs simply don’t have infinite context lengths.

Auto-Merging is a retrieval technique that leverages a hierarchical document structure. When a document is too long, it is split into smaller documents or chunks, where we can think of the smaller documents as the children of the original document and the original document as the parent. This results in a hierarchical tree structure where each smaller document is a child of a previous larger document. The leaves of the tree are the documents which don’t have any children, and the root is the original document.

Haystack 2.5.1

Tue, 10 Sep 2024 00:00:00 +0000

⚡️ Enhancement Notes

Add default_headers init argument to AzureOpenAIGenerator and AzureOpenAIChatGenerator

🐛 Bug Fixes

Fix the Pipeline visualization issue due to changes in the new release of Mermaid
Fix Pipeline not running Components with Variadic input even if it received inputs only from a subset of its senders
The from_dict method of ConditionalRouter now correctly handles the case where the dict passed to it contains the key custom_filters explicitly set to None. Previously this was causing an AttributeError

Haystack 2.5.0

Wed, 04 Sep 2024 00:00:00 +0000

⬆️ Upgrade Notes

Removed ChatMessage.to_openai_format method. Use haystack.components.generators.openai_utils._convert_message_to_openai_format instead.
Removed unused debug parameter from Pipeline.run method.
Removed deprecated SentenceWindowRetrieval. Use SentenceWindowRetriever instead.

🚀 New Features

Added the unsafe argument to enable behavior that could lead to remote code execution in ConditionalRouter and OutputAdapter. By default, unsafe behavior is disabled, and users must explicitly set unsafe=True to enable it. When unsafe is enabled, types such as ChatMessage, Document, and Answer can be used as output types. We recommend enabling unsafe behavior only when the Jinja template source is trusted. For more information, see the documentation for ConditionalRouter and OutputAdapter.

⚡️ Enhancement Notes

Adapts how ChatPromptBuilder creates ChatMessages. Messages are deep copied to ensure all meta fields are copied correctly.
The parameter, min_top_k, has been added to the TopPSampler. This parameter sets the minimum number of documents to be returned when the top-p sampling algorithm selects fewer documents than desired. Documents with the next highest scores are added to meet the minimum. This is useful when guaranteeing a set number of documents to pass through while still allowing the Top-P algorithm to determine if more documents should be sent based on scores.
Introduced a utility function to deserialize a generic Document Store from the init_parameters of a serialized component.
Refactor deserialize_document_store_in_init_parameters to clarify that the function operates in place and does not return a value.
The SentenceWindowRetriever now returns context_documents as well as the context_windows for each Document in retrieved_documents . This allows you to get a list of Documents from within the context window for each retrieved document.

⚠️ Deprecation Notes

The default model for OpenAIGenerator and OpenAIChatGenerator, previously ‘gpt-3.5-turbo’, will be replaced by ‘gpt-4o-mini’.

🐛 Bug Fixes

Fixed an issue where page breaks were not being extracted from DOCX files.
Used a forward reference for the Paragraph class in the DOCXToDocument converter to prevent import errors.
The metadata produced by DOCXToDocument component is now JSON serializable. Previously, it contained datetime objects automatically extracted from DOCX files, which are not JSON serializable. These datetime objects are now converted to strings.
Starting from haystack-ai==2.4.0, Haystack is compatible with sentence-transformers>=3.0.0; earlier versions of sentence-transformers are not supported. We have updated the test dependencies and LazyImport messages to reflect this change.
For components that support multiple Document Stores, prioritize using the specific from_dict class method for deserialization when available. Otherwise, fall back to the generic default_from_dict method. This impacts the following generic components: CacheChecker, DocumentWriter, FilterRetriever, and SentenceWindowRetriever.

Announcing Our New Short Course with DeepLearning.AI

Wed, 21 Aug 2024 00:00:00 +0000

DeepLearning.AI and Andrew Ng hold a special place in many AI/ML engineers’ careers and development. For many engineers, they’ve played a pivotal role into breaking into AI, or extending their knowledge and capabilities by teaching about core concepts and technologies at the right time. So, we’re incredibly happy to announce that we’re launching our first short course with them - “Building AI Applications with Haystack”

🚀 Enroll now

Haystack 2.4.0

Thu, 15 Aug 2024 00:00:00 +0000

Highlights

🙌 Local LLMs and custom generation parameters in evaluation

The new api_params init parameter added to LLM-based evaluators such as ContextRelevanceEvaluator and FaithfulnessEvaluator can be used to pass in supported OpenAIGenerator parameters, allowing for custom generation parameters (via generation_kwargs) and local LLM support (via api_base_url).

📝 New Joiner

New AnswerJoiner component to combine multiple lists of Answers.

⬆️ Upgrade Notes

The ContextRelevanceEvaluator now returns a list of relevant sentences for each context, instead of all the sentences in a context. Also, a score of 1 is now returned if a relevant sentence is found, and 0 otherwise.
Removed the deprecated DynamicPromptBuilder and DynamicChatPromptBuilder components. Use PromptBuilder and ChatPromptBuilder instead.
OutputAdapter and ConditionalRouter can’t return users inputs anymore.
Multiplexer is removed and users should switch to BranchJoiner instead.
Removed deprecated init parameters extractor_type and try_others from HTMLToDocument.
SentenceWindowRetrieval component has been renamed to SenetenceWindowRetriever.
The serialize_callback_handler and deserialize_callback_handler utility functions have been removed. Use serialize_callable and deserialize_callable instead. For more information on serialize_callable and deserialize_callable, see the API reference: https://docs.haystack.deepset.ai/reference/utils-api#module-callable_serialization

🚀 New Features

LLM based evaluators can pass in supported OpenAIGenerator parameters via api_params. This allows for custom generation_kwargs, changing the api_base_url (for local evaluation), and all other supported parameters as described in the OpenAIGenerator docs.
Introduced a new AnswerJoiner component that allows joining multiple lists of Answers into a single list using the Concatenate join mode.
Add truncate_dim parameter to Sentence Transformers Embedders, which allows truncating embeddings. Especially useful for models trained with Matryoshka Representation Learning.
Add precision parameter to Sentence Transformers Embedders, which allows quantized embeddings. Especially useful for reducing the size of the embeddings of a corpus for semantic search, among other tasks.

⚡️ Enhancement Notes

Adds model_kwargs and tokenizer_kwargs to the components TransformersSimilarityRanker, SentenceTransformersDocumentEmbedder, SentenceTransformersTextEmbedder. This allows passing things like model_max_length or torch_dtype for better management of model inference.
Added unicode_normalization parameter to the DocumentCleaner, allowing to normalize the text to NFC, NFD, NFKC, or NFKD.
Added ascii_only parameter to the DocumentCleaner, transforming letters with diacritics to their ASCII equivalent and removing other non-ASCII characters.
Improved error messages for deserialization errors.
TikaDocumentConverter now returns page breaks (“f”) in the output. This only works for PDF files.
Enhanced filter application logic to support merging of filters. It facilitates more precise retrieval filtering, allowing for both init and runtime complex filter combinations with logical operators. For more details see https://docs.haystack.deepset.ai/docs/metadata-filtering
The streaming_callback parameter can be passed to OpenAIGenerator and OpenAIChatGenerator during pipeline run. This prevents the need to recreate pipelines for streaming callbacks.
Add max_retries and timeout parameters to the AzureOpenAIChatGenerator initializations.
Document Python 3.11 and 3.12 support in project configuration.
Refactor DocumentJoiner to use enum pattern for the ‘join_mode’ parameter instead of bare string.
Add max_retries, timeout parameters to the AzureOpenAIDocumentEmbedder initialization.
Add max_retries and timeout parameters to the AzureOpenAITextEmbedder initializations.
Introduce an utility function to deserialize a generic Document Store from the init_parameters of a serialized component.

⚠️ Deprecation Notes

Haystack 1.x legacy filters are deprecated and will be removed in a future release. Please use the new filter style as described in the documentation - https://docs.haystack.deepset.ai/docs/metadata-filtering
Deprecate the method to_openai_format of the ChatMessage dataclass. This method was never intended to be public and was only used internally. Now, each Chat Generator will know internally how to convert the messages to the format of their specific provider.
Deprecate the unused debug parameter in the Pipeline.run method.
SentenceWindowRetrieval is deprecated and will be removed in future. Use SentenceWindowRetriever instead.

Security Notes

Fix issue that could lead to remote code execution when using insecure Jinja template in the following Components:

Advanced RAG: Query Expansion

Wed, 14 Aug 2024 00:00:00 +0000

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval

2️⃣ Query Expansion

3️⃣ Query Decomposition

4️⃣ Automated Metadata Enrichment

The quality of RAG (retrieval augmented generation) highly depends on the quality of the first step in the process: retrieval. The generation step can only be as good as the context its working on, which it will receive as a result of a retrieval step.

CPU-Optimized Embedding Models with fastRAG and Haystack

Thu, 01 Aug 2024 00:00:00 +0000

One of the main and critical components of a retrieval augmented generation (RAG) pipeline is the embedding process, which forms the foundation for efficient information retrieval by transforming raw text into machine-readable vector representations. Embedding models encode textual data into dense vectors, capturing semantic and contextual meaning. These models are used to create embeddings for both queries (for retrieval) and documents (for indexing and reranking). Therefore, optimizing these models through quantization could improve our RAG application by providing:

Haystack 2.3.1

Mon, 29 Jul 2024 00:00:00 +0000

⬆️ Upgrade Notes

OutputAdapter and ConditionalRouter can’t return users inputs anymore.

Security Notes

Fix issue that could lead to remote code execution when using insecure Jinja template in the following Components:
- PromptBuilder
- ChatPromptBuilder
- DynamicPromptBuilder
- DynamicChatPromptBuilder
- OutputAdapter
- ConditionalRouter
The same issue has been fixed in the PipelineTemplate class too.

Mixedbread 🤝 deepset: Announcing our New German/English Embedding Model

Thu, 18 Jul 2024 00:00:00 +0000

It’s 2024 and yet, most models today are still primarily geared towards English speaking markets. Today, deepset and Mixedbread are jointly announcing our latest contribution towards changing that landscape: A new open-source German/English embedding model - deepset-mxbai-embed-de-large-v1.

Our model is based on intfloat/multilingual-e5-large and was fine-tuned on 30+ million pairs of German data for retrieval tasks. On the NDCG@10 metric, which compares the list of retrieval results against an ideally ordered list of expected results, our model not only sets a new standard for open-source German embedding models but is also competitive with commercial alternatives.

Haystack 2.3.0

Mon, 15 Jul 2024 00:00:00 +0000

🧑‍🔬 Haystack Experimental Package

Alongside this release, we’re introducing a new repository and package: haystack-experimental. This package will be installed alongside haystack-ai and will give you access to experimental components. As the name suggests, these components will be highly exploratory, and may or may not make their way into the main haystack package.

Each experimental component in the haystack-experimental repo will have a life-span of 3 months
The end of the 3 months marks the end of the experiment. In which case the component will either move to the core haystack package, or be discontinued

To learn more about the experimental package, check out the Experimental Package docs[LINK] and the API references[LINK] To use components in the experimental package, simply from haystack_experimental.component_type import Component What’s in there already?

Haystack 2.2.4

Thu, 04 Jul 2024 00:00:00 +0000

⚡️ Enhancement Notes

Added the apply_filter_policy function to standardize the application of filter policies across all document store-specific retrievers, allowing for consistent handling of initial and runtime filters based on the chosen policy (replace or merge).
Introduced a ‘filter_policy’ init parameter for both InMemoryBM25Retriever and InMemoryEmbeddingRetriever, allowing users to define how runtime filters should be applied with options to either ‘replace’ the initial filters or ‘merge’ them, providing greater flexibility in filtering query results.

🐛 Bug Fixes

Meta handling of bytestreams in Azure OCR has been fixed.
Fix some bugs running a Pipeline that has Components with conditional outputs. Some branches that were expected not to run would run anyway, even if they received no inputs. Some branches instead would cause the Pipeline to get stuck waiting to run that branch, even if they received no inputs. The behaviour would depend whether the Component not receiving the input has a optional input or not.

Benchmarking Haystack Pipelines for Optimal Performance

Mon, 24 Jun 2024 00:00:00 +0000

In this article, we will show you how to use Haystack to evaluate the performance of a RAG pipeline. Note that the code in this article is meant to be illustrative and may not run as is; if you want to run the code, please refer to the python script.

Introduction

This article will guide you through building a Retrieval-Augmented Generation (RAG) pipeline using Haystack, adjusting various parameters, and evaluating it with the ARAGOG dataset. The dataset consists of pairs of questions and answers, and our objective is to assess the RAG pipeline’s efficiency in retrieving the correct context and generating accurate answers. To do this, we will use the following evaluation metrics:

Haystack 2.2.3

Mon, 17 Jun 2024 00:00:00 +0000

🐛 Bug Fixes

Pin numpy<2 to avoid breaking changes that cause several core integrations to fail. Pin tenacity too (8.4.0 is broken).

⚡️ Enhancement Notes

Export ChatPromptBuilder in builders module

RAG Evaluation with Prometheus 2

Mon, 17 Jun 2024 00:00:00 +0000

When building real-world applications based on Language Models (such as RAG), evaluation plays an important role. Recently, evaluating generated answers using powerful proprietary Language Models (such as GPT-4) has become popular and correlates well with human judgment, but it comes with its own limitations and challenges.

Prometheus 2 is a newly released family of open-source models specifically trained to evaluate the output of other Language Models. In this article (and in the related notebook), we will see how to use Prometheus and we will experiment with it to evaluate the generated responses of a RAG Pipeline using Haystack.

Haystack 2.2.2

Fri, 07 Jun 2024 00:00:00 +0000

🐛 Bug Fixes

Add missing metrics column in DataFrame returned by EvaluationRunResult.score_report()

Haystack 2.2.1

Thu, 06 Jun 2024 00:00:00 +0000

⬆️ Upgrade Notes

trafilatura must now be manually installed with pip install trafilatura to use the HTMLToDocument Component.

⚡️ Enhancement Notes

Remove trafilatura as direct dependency and make it a lazily imported one

Haystack 2.2.0

Mon, 03 Jun 2024 00:00:00 +0000

Highlights

The Multiplexer component proved to be hard to explain and to understand. After reviewing its use cases, the documentation was rewritten and the component was renamed to BranchJoiner to better explain its functionalities.

Add the ‘OPENAI_TIMEOUT’ and ‘OPENAI_MAX_RETRIES’ to the OpenAI components.

⬆️ Upgrade Notes

BranchJoiner has the very same interface asMultiplexer. To upgrade your code, just rename any occurrence ofMultiplexer toBranchJoiner and ajdust the imports accordingly.

🚀 New Features

AddBranchJoiner to eventually replaceMultiplexer
AzureOpenAIGenerator andAzureOpenAIChatGenerator can now be configured passing a timeout for the underlyingAzureOpenAI client.

⚡️ Enhancement Notes

ChatPromptBuilder now supports changing its template at runtime. This allows you to define a default template and then change it based on your needs at runtime.
If an LLM-based evaluator (e.g.,Faithfulness orContextRelevance) is initialised withraise_on_failure=False, and if a call to an LLM fails or an LLM outputs an invalid JSON, the score of the sample is set toNaN instead of raising an exception. The user is notified with a warning indicating the number of requests that failed.
Adds inference mode to model call of the ExtractiveReader. This prevents gradients from being calculated during inference time in pytorch.
TheDocumentCleaner class has the optional attributekeep_id that if set to True it keeps the document ids unchanged after cleanup.
DocumentSplitter now has an optional split_threshold parameter. Use this parameter if you want to rather not split inputs that are only slightly longer than the allowed split_length. If when chunking one of the chunks is smaller than the split_threshold, the chunk will be concatenated with the previous one. This avoids having too small chunks that are not meaningful.
Re-implementInMemoryDocumentStore BM25 search with incremental indexing by avoiding re-creating the entire inverse index for every new query. This change also removes the dependency onhaystack_bm25. Please refer to [PR #7549]( https://github.com/deepset-ai/haystack/pull/7549) for the full context.
Improved MIME type management by directly setting MIME types on ByteStreams, enhancing the overall handling and routing of different file types. This update makes MIME type data more consistently accessible and simplifies the process of working with various document formats.
PromptBuilder now supports changing its template at runtime (e.g. for Prompt Engineering). This allows you to define a default template and then change it based on your needs at runtime.
Now you can set the timeout and max_retries parameters on OpenAI components by setting the ‘OPENAI_TIMEOUT’ and ‘OPENAI_MAX_RETRIES’ environment vars or passing them at __init__.
TheDocumentJoiner component’srun method now accepts atop_k parameter, allowing users to specify the maximum number of documents to return at query time. This fixes issue #7702.
Enforce JSON mode on OpenAI LLM-based evaluators so that the they always return valid JSON output. This is to ensure that the output is always in a consistent format, regardless of the input.
Makewarm_up() usage consistent across the codebase.
Create a class hierarchy for pipeline classes, and move the run logic into the child class. Preparation work for introducing multiple run stratgegies.
Make theSerperDevWebSearch more robust whensnippet is not present in the request response.
Make SparseEmbedding a dataclass, this makes it easier to use the class with Pydantic
`HTMLToDocument`: change the HTML conversion backend fromboilerpy3 totrafilatura, which is more robust and better maintained.

⚠️ Deprecation Notes

Mulitplexer is now deprecated.
DynamicChatPromptBuilder has been deprecated asChatPromptBuilder fully covers its functionality. UseChatPromptBuilder instead.
DynamicPromptBuilder has been deprecated asPromptBuilder fully covers its functionality. UsePromptBuilder instead.
The following parameters ofHTMLToDocument are ignored and will be removed in Haystack 2.4.0:extractor_type andtry_others.

🐛 Bug Fixes

FaithfullnessEvaluator andContextRelevanceEvaluator now return0 instead ofNaN when applied to an empty context or empty statements.
Azure generators components fixed, they were missing the@component decorator.
Updates the from_dict method of SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder, NamedEntityExtractor, SentenceTransformersDiversityRanker and LocalWhisperTranscriber to allow None as a valid value for device when deserializing from a YAML file. This allows a deserialized pipeline to auto-determine what device to use using the ComponentDevice.resolve_device logic.
Fix the broken serialization of HuggingFaceAPITextEmbedder, HuggingFaceAPIDocumentEmbedder, HuggingFaceAPIGenerator, and HuggingFaceAPIChatGenerator.
FixNamedEntityExtractor crashing in Python 3.12 if constructed using a string backend argument.
Fixed the PdfMinerToDocument converter’s outputs to be properly wired up to ‘documents’.
Addto_dict method toDocumentRecallEvaluator to allow proper serialization of the component.
Improves/fixes type serialization of PEP 585 types (e.g. list[Document], and their nested version). This improvement enables better serialization of generics and nested types and improves/fixes matching of list[X] and List[X] types in component connections after serialization.
Fixed (de)serialization of NamedEntityExtractor. Includes updated tests verifying these fixes when NamedEntityExtractor is used in pipelines.
Theinclude_outputs_from parameter inPipeline.run correctly returns outputs of components with multiple outputs.
Return an empty list of answers whenExtractiveReader receives an empty list of documents instead of raising an exception.

Building RAG Applications with NVIDIA NIM and Haystack on K8s

Sun, 02 Jun 2024 00:00:00 +0000

Retrieval-augmented generation (RAG) systems combine generative AI with information retrieval for contextualized answer generation. Building reliable and performant RAG applications at scale is challenging. In this blog, we show how to use Haystack and NVIDIA NIM to create a RAG solution which is easy to deploy/maintain, standardized and enterprise-ready, that can run on-prem as well as on cloud native environments. This recipe is applicable in the cloud, on-premise or even in air-gapped environments.

Monitor and trace your Haystack pipelines with Langfuse

Fri, 17 May 2024 00:00:00 +0000

Getting your LLM application into production is a huge milestone, but that’s only the beginning. It’s critical to monitor how your pipeline is performing in the real world so you can keep improving performance and cost, and proactively address any issues that might arise.

With the new Haystack Langfuse integration, it’s now easier than ever to have visibility into your pipelines. In this post, we’ll explain more about Langfuse, and demonstrate how to trace an end to end request to a Haystack pipeline.

Haystack 2.1.2

Thu, 16 May 2024 00:00:00 +0000

⚡️ Enhancement Notes

Enforce JSON mode on OpenAI LLM-based evaluators so that they always return valid JSON output. This is to ensure that the output is always in a consistent format, regardless of the input.

🐛 Bug Fixes

FaithfullnessEvaluator and ContextRelevanceEvaluator now return 0 instead of NaN when applied to an empty context or empty statements.
Azure generators components fixed, they were missing the @component decorator.
Updates the from_dict method of SentenceTransformersTextEmbedder, SentenceTransformersDocumentEmbedder, NamedEntityExtractor, SentenceTransformersDiversityRanker and LocalWhisperTranscriber to allow None as a valid value for device when deserializing from a YAML file. This allows a deserialized pipeline to auto-determine what device to use using the ComponentDevice.resolve_device logic.
Improves/fixes type serialization of PEP 585 types (e.g. list[Document], and their nested version). This improvement enables better serialization of generics and nested types and improves/fixes matching of list[X] and List[X]` types in component connections after serialization.
Fixed (de)serialization of NamedEntityExtractor. Includes updated tests verifying these fixes when NamedEntityExtractor is used in pipelines.
The include_outputs_from parameter in Pipeline.run correctly returns outputs of components with multiple outputs.

Advanced Retrieval: Extract Metadata from Queries to Improve Retrieval

Mon, 13 May 2024 00:00:00 +0000

This is part one of the Advanced Use Cases series:

1️⃣ Extract Metadata from Queries to Improve Retrieval

2️⃣ Query Expansion

3️⃣ Query Decomposition

4️⃣ Automated Metadata Enrichment

In Retrieval-Augmented Generation (RAG) applications, the retrieval step, which provides relevant context to your large language model (LLM), is vital for generating high-quality responses. There are possible ways of improving retrieval and metadata filtering is one of the easiest ways. Metadata filtering, the approach of limiting the search space based on some concrete metadata, can really enhance the quality of the retrieved documents. Here are some advantages of using metadata filtering:

Haystack 2.1.1

Thu, 09 May 2024 00:00:00 +0000

⚡️ Enhancement Notes

Make SparseEmbedding a dataclass, this makes it easier to use the class with Pydantic

🐛 Bug Fixes

Fix the broken serialization of HuggingFaceAPITextEmbedder, HuggingFaceAPIDocumentEmbedder, HuggingFaceAPIGenerator, and HuggingFaceAPIChatGenerator.
Add to_dict method to DocumentRecallEvaluator to allow proper serialization of the component.

Haystack 2.1.0

Tue, 07 May 2024 00:00:00 +0000

Highlights

📊 New Evaluator Components

Haystack introduces new components for both with model-based, and statistical evaluation: AnswerExactMatchEvaluator, ContextRelevanceEvaluator, DocumentMAPEvaluator, DocumentMRREvaluator, DocumentRecallEvaluator, FaithfulnessEvaluator, LLMEvaluator, SASEvaluator

Here’s an example of how to use DocumentMAPEvaluator to evaluate retrieved documents and calculate mean average precision score:

from haystack import Document
from haystack.components.evaluators import DocumentMAPEvaluator

evaluator = DocumentMAPEvaluator()
result = evaluator.run(
    ground_truth_documents=[
        [Document(content="France")],
        [Document(content="9th century"), Document(content="9th")],
    ],
    retrieved_documents=[
        [Document(content="France")],
        [Document(content="9th century"), Document(content="10th century"), Document(content="9th")],
    ],
)

result["individual_scores"]
>> [1.0, 0.8333333333333333]
result["score"]
>> 0 .9166666666666666

To learn more about evaluating RAG pipelines both with model-based, and statistical metrics available in the Haystack, check out Tutorial: Evaluating RAG Pipelines.

Chatting with SQL Databases 3 Ways

Mon, 22 Apr 2024 00:00:00 +0000

Let’s talk about how we build AI applications that can interact with, even chat to SQL databases. Heads-up that this is a mini project that I tried out, and it’s objectively a simple approach to interacting with SQL with natural language. I’m certain that this is not a golden bullet that works with all types of SQL tables etc. But, nonetheless, it’s cool, it works, and you can try it along with me.

Haystack 2.0.1

Tue, 09 Apr 2024 00:00:00 +0000

⬆️ Upgrade Notes

The HuggingFaceTGIGenerator and HuggingFaceTGIChatGenerator components have been modified to be compatible with huggingface_hub>=0.22.0.

If you use these components, you may need to upgrade the huggingface_hub library. To do this, run the following command in your environment: pip install "huggingface_hub>=0.22.0"

🚀 New Features

Adds streaming_callback parameter to HuggingFaceLocalGenerator, allowing users to handle streaming responses.
Introduce a new SparseEmbedding class which can be used to store a sparse vector representation of a Document. It will be instrumental to support Sparse Embedding Retrieval with the subsequent introduction of Sparse Embedders and Sparse Embedding Retrievers.

⚡️ Enhancement Notes

Set max_new_tokens default to 512 in Hugging Face generators.

Level up Your RAG Application with Speaker Diarization

Thu, 21 Mar 2024 00:00:00 +0000

LLMs work wonders on text data. Using LLMs, you can get answers to complex questions on long documents without having to read the document or even do a CTRL+F search. But what if you work with audio or video recordings?

The easiest way is to provide the LLM with the transcription of the recording. That way you can capture everything that is uttered in the audio or video. But what about information that was not spoken?

Hosted or self-hosted RAG? Full flexibility with NVIDIA NIM integration in Haystack

Mon, 18 Mar 2024 00:00:00 +0000

Iteration speed across design options matters

Retrieval-augmented generation (RAG) is one of the most common architectures today for building LLM applications. With RAG you can very quickly build a prototype that is tailored to your use case by connecting the LLM to the relevant data sources.

Once you have your first prototype up and running, you typically iterate a lot on your pipeline design before you go live in production: switching embedding models or generative LLMs, adding rerankers, or leveraging the metadata of your documents.

Haystack 2.0: The Composable Open-Source LLM Framework

Mon, 11 Mar 2024 00:00:00 +0000

Today we are happy to announce the stable release of Haystack 2.0 - we’ve been working on this for a while, and some of you have already been testing the beta since its first release in December 2023.

Haystack is an open-source Python framework for building production-ready LLM applications, with integrations to almost all major model providers and databases.

At its core, Haystack 2.0 is a major rework of the previous version with a very clear goal in mind: making it possible to implement composable AI systems that are easy to use, customize, extend, optimise, evaluate and ultimately deploy to production.

Haystack 2.0.0

Mon, 11 Mar 2024 00:00:00 +0000

Today, we’ve released the stable version of Haystack 2.0. This is ultimately a rewrite of the Haystack framework, so these release notes are not what you’d usually expect to see in regular release notes where we highlight specific changes to the codebase. Instead, we will highlight features of Haystack 2.0 and how it’s meant to be used.

To read more about our motivation for Haystack 2.0 and what makes up our design choices, you can read our release announcement article.

Optimizing Retrieval with HyDE

Wed, 28 Feb 2024 00:00:00 +0000

Hypothetical Document Embeddings (HyDE) is a technique proposed in the paper “ Precise Zero-Shot Dense Retrieval without Relevance Labels” which improves retrieval by generating “fake” hypothetical documents based on a given query, and then uses those “fake” documents embeddings to retrieve similar documents from the same embedding space.

In this article, we will see how to implement and incorporate it into Haystack by creating a custom component that implements HyDE.

To learn more about how HyDE works, and where it’s useful, check out our guide on Hypothetical Document Embeddings (HyDE)

Announcing the Astra DB Haystack Integration

Fri, 19 Jan 2024 00:00:00 +0000

The Haystack extension family is growing so fast, it’s hard to keep up! Our latest addition is the Astra DB extension by Datastax. It’s an open source package that helps you use Astra DB as a vector database for your Haystack pipelines.

Let’s learn about the benefits of Astra DB and how to use it with Haystack.

Benefits of Astra DB

DataStax Astra DB is a serverless vector database built on Apache Cassandra. What makes Astra DB special?

PDF-Based Question Answering with Amazon Bedrock and Haystack

Wed, 17 Jan 2024 00:00:00 +0000

In this article, I’ll guide you through the process of creating a generative question answering application tailored for PDF files using the newly added Amazon Bedrock integration with Haystack and OpenSearch to store our documents efficiently. The demo will illustrate the step-by-step development of a QA application designed specifically for the Bedrock documentation, demonstrating the power of Bedrock in the process 🚀

Using Jina Embeddings v2 with Haystack pipelines to summarize legal documents

Wed, 10 Jan 2024 00:00:00 +0000

Jina.ai recently upgraded and expanded the capabilities of their previous embedding model in a v2 release.

With the Jina Haystack extension, you can now take advantage of these new text embedders in your Haystack pipelines! In this post, we’ll show what’s cool about Jina Embeddings v2 and how to use them.

You can follow along in the accompanying Colab notebook of a RAG pipeline that uses the Jina Haystack extension.

Advantages of Jina Embeddings v2

Handling long documents. The large token window, accommodating up to 8192 tokens, allows you to break the embeddings into larger chunks. It’s more computationally and memory-efficient to use a few larger vectors than a lot of small ones, so this allows Jina v2 to process large documents efficiently.
Improved semantic understanding. Larger text chunks also contain more context within each chunk, which can help LLMs better understand your documents. Improved understanding means better long document retrieval, semantic textual similarity, text reranking, recommendation, RAG and LLM-based generative search.
Short vector length: Jina Embeddings v2 emits embedding vectors of length 768 (base model) or 512 (small model), which are both significantly less than that of the only other embedding model that supports 8k tokens input length, while not compromising on the quality of retrieval, similarity, reranking or other downstream tasks. A shorter vector length implies cost-savings for the vector database, which typically price based on stored vector dimensions.
Fully open source 💙 There are both small and large embedding models available, depending on your computing resources and requirements. To run the embedding models yourself, check out this documentation on HuggingFace. Alternately, you can use Jina’s fully managed embedding service to handle that for you, which we’ll be doing for this demo.

Getting started using Jina Embeddings v2 with Haystack

To use the integration you’ll need a free Jina api key - get one here.

Highlights of 2023

Fri, 05 Jan 2024 00:00:00 +0000

This post was originally shared through the Haystack newsletter. Subscribe now to stay in the loop on all things Haystack, including the latest updates, new features, captivating content, and upcoming exciting events! 🗞️

What a fantastic year it has been! In 2023, the world of AI saw tremendous progress, making it an exciting time for learning and growth. As we wind down and approach the new year, it’s a good moment to take a breather, look back on the past year, and appreciate the highlights before gearing up for what’s ahead. We’ve taken some time to review the noteworthy moments that made 2023 special for Haystack and its community.

Building a Healthcare Chatbot with Mixtral 8x7b, Haystack, and PubMed

Tue, 02 Jan 2024 00:00:00 +0000

Unfortunately, countless people around the world have inadequate access to healthcare. I’m lucky to have health insurance and good medical providers taking care of me. However, I still want to educate myself before walking into a doctor’s office.

Technology can empower people to take charge of their health. Large language models can power chatbots where people can ask medical questions.

In this post, I’ll show you how I built a medical chatbot with Haystack 2.0-Beta, and the Mixtral 8x7B model by pulling research papers from PubMed.

Gemini Models with Google Vertex AI Integration for Haystack

Mon, 18 Dec 2023 00:00:00 +0000

In this article, we will introduce you to the new Google Vertex AI Integration for Haystack. While this integration introduces several new components to the Haystack eco-system (feel free to explore the full integration repo!), we’d like to start by showcasing two components in particular: the VertexAIGeminiGenerator and the VertexAIGeminiChatGenerator, using the gemini-pro and gemini-1.5-flash models.

💚 You can run the example code showcased in this article in the accompanying Notebook

The great news is, to authenticate for access to the Gemini models, you will only need to do a Google authentication in the Colab (instructions in the Colab)

Use Gradient Models with Haystack Pipelines

Mon, 11 Dec 2023 00:00:00 +0000

One of the more cumbersome aspects of creating LLM applications is model management. Especially in cases where we need to fine-tune, host, and scale the models ourselves. In this case, having options at hand can be great. Today, we’ve expanded the Haystack 2.0 ecosystem with a new integration that can help you with just that

Gradient is an LLM development platform that offers web APIs for fine-tuning, embeddings, and inference on state-of-the-art open-source models. In this article, let’s take a look at the new Gradient integration for Haystack, and how you can use it in your retrieval-augmented generative pipelines.

Introducing Haystack 2.0-Beta and Advent of Haystack

Mon, 04 Dec 2023 00:00:00 +0000

Today, we are really happy to announce that we have released Haystack 2.0-Beta, alongside our first-ever Advent of Haystack: a set of 10 challenges that we will publish throughout the month of December, each introducing the features and design of Haystack 2.0-Beta.

What does this release mean for me?

Since the first day we started building Haystack 2.0, we’ve involved our community with our design decisions and the feedback we got on our proposals on GitHub and via our Discord community proved to be incredibly valuable. While this is not yet the full stable release of Haystack 2.0, we want to make this first official commitment to the new design available for you to test and truly experience how Haystack is improving. We are committed to redesigning our LLM framework, and we need your help to shape it. To participate, complete and submit a challenge, with any feedback you would like to give us about your experience.

Using Generative AI to Query Large BI Tables: Our Findings

Wed, 29 Nov 2023 00:00:00 +0000

In organizations with large collections of data in tabular form, it’s the job of a data analyst to make sense of that data by extracting, transforming, and building stories around it. An analyst’s primary tool for accessing the data is SQL. Given the impressive capabilities of large language models (LLMs), it’s natural to wonder if AI can help us translate our information needs into well-formed SQL queries.

Granted, most LLMs can output at least some SQL queries based on natural language input. But can they handle large, multi-table databases in a real-world setting? In other words, can they do the job of a data analyst? A small team here at deepset set out to answer that question. Over the course of three months, we tried to find the best way to generate SQL queries on a real dataset.

RAG Pipelines From Scratch

Tue, 21 Nov 2023 00:00:00 +0000

Retrieval Augmented Generation (RAG) is quickly becoming an essential technique to make LLMs more reliable and effective at answering any question, regardless of how specific. To stay relevant in today’s NLP landscape, Haystack must enable it.

Let’s see how to build such applications with Haystack 2.0, from a direct call to an LLM to a fully-fledged, production-ready RAG pipeline that scales. At the end of this post, we will have an application that can answer questions about world countries based on data stored in a private database. At that point, the knowledge of the LLM will be only limited by the content of our data store, and all of this can be accomplished without fine-tuning language models.

Multilingual Generative Question Answering with Haystack and Cohere

Wed, 08 Nov 2023 00:00:00 +0000

In today’s interconnected world, language should never be a barrier to accessing information. This is especially crucial in the context of travel, where travelers often rely on reviews to make informed decisions about their accommodations (I know I do). But what if you’re an English-speaking traveler trying to make sense of reviews written in multiple languages and simply want to know, “Is this place too noisy to sleep?”

In this blog post, we’ll delve into the details of multilingual retrieval and multilingual generation, and demonstrate how to build a Retrieval Augmented Generation (RAG) pipeline to generate answers from multilingual hotel reviews using Cohere models and Haystack.

Guide to Using Zephyr Models to Generate Answers on Your Data

Mon, 06 Nov 2023 00:00:00 +0000

Hugging Face recently announced their new open-source LLM, Zephyr-7B Beta, which is a fine-tuned version of Mistral 7B V.01 that focuses on helpfulness and outperforms many larger models on MT-Bench and AlpacaEval benchmarks. In this article, we’re going to show you how to use the new Zephyr models in a full retrieval-augmented generation pipeline, in a way that can work on your own private data.

Following the theme of our previous article, we will show you how to build a pipeline that uses Zephyr with Haystack, but we will also take the opportunity and show you how to do this with the preview package of Haystack 2.0.

Customizing RAG Pipelines to Summarize Latest Hacker News Posts

Fri, 22 Sep 2023 00:00:00 +0000

Over the last few months, the team at deepset has been working on a major upgrade in the Haystack repository. Along the way, we’ve been sharing our updates and design process for the upcoming Haystack 2.0 with the community, as well as releasing new components in a preview package. This means that you can already start exploring features coming to Haystack 2.0 using the preview components available in the haystack-ai package (pip install haystack-ai).

Deploying RAG to Production

Wed, 13 Sep 2023 00:00:00 +0000

As data scientists, we have often mastered the art of prototyping. We can use machine learning frameworks like Haystack to build, test, and fine-tune data-driven systems. We’re experts at gathering stakeholder feedback, quantifying it, and interpreting the various performance metrics. But it can get tricky for many of us when we want to move these systems into a production environment, where they become available for general use.

With Haystack, developers can build complex LLM pipelines on top of their own text databases, using state-of-the-art tools: from conversational AI to semantic search and summarization. One of the most talked about architectures these days is RAG, which stands for retrieval augmented generative AI. RAG pipelines combine the power of a generative LLM with the insights contained in your data, to create truly helpful user interfaces. To learn more, check out our blog post on RAG on The Deep Dive.

Talk to YouTube Videos with Haystack Pipelines

Fri, 08 Sep 2023 00:00:00 +0000

You can use this Colab for a working example of the application described in this article.

In this article, I’ll be showing an example of how to leverage transcription models like OpenAI’s Whisper, so as to build a retrieval augmented generation (RAG) pipeline that will allow us to effectively search through video content.

The example application I’ll showcase is able to answer questions based on the transcript extracted from the video. I’ll use the video by Erika Cardenas as an example. In the video, she talks about chunking and preprocessing documents for RAG pipelines. Once we’re done, we will be able to query a Haystack pipeline that will respond based on the contents of the video.

Talk to Haystack Docs: Creating a Domain-Focused Q&A RAG Pipeline with WebRetriever

Mon, 04 Sep 2023 00:00:00 +0000

Ever since its introduction, WebRetriever has proven useful in the Haystack ecosystem. As its name implies, WebRetriever allows fetching documents from the Internet and channelling them into Haystack pipelines.

Under the hood, WebRetriever uses a search engine to look up relevant documents to retrieve from the web. Although users could customize the domain scope of the search queries even before (using the “site:” AND/OR syntax mixed with the actual query), it often felt like a workaround rather than a feature. Even worse – it created a barrier for users to exploit the WebRetriever capabilities fully. We needed to make it more intuitive and less “hacky”.

Generative Documentation Q&A with Weaviate and Haystack

Sat, 02 Sep 2023 00:00:00 +0000

You can use this Colab for a working example of the application described in this article.

Retrieval augmented generation is the golden child of LLM applications lately. The idea behind it is simple: LLMs do not know the entire world, least of all your specific world. But, with the use of retrieval techniques, we can provide the most useful pieces of information to an LLM so that it has the context with which to reply to queries that it otherwise would not have been trained to know about or answer.

Enhancing RAG Pipelines in Haystack: Introducing DiversityRanker and LostInTheMiddleRanker

Tue, 29 Aug 2023 00:00:00 +0000

The recent improvements in Natural Language Processing (NLP) and Long-Form Question Answering (LFQA) would have, just a few years ago, sounded like something from the domain of science fiction. Who could have thought that nowadays we would have systems that can answer complex questions with the precision of an expert, all while synthesizing these answers on the fly from a vast pool of sources? LFQA is a type of Retrieval-Augmented Generation (RAG) which has recently made significant strides, utilizing the best retrieval and generation capabilities of Large Language Models (LLMs).

Hybrid Document Retrieval

Tue, 22 Aug 2023 00:00:00 +0000

Document retrieval is the art of extracting relevant documents from a corpus in response to an input. Like many language-related tasks today, it can benefit greatly from the dense, semantic embeddings produced by encoder models. These models have learned to embed documents in an abstract vector space that captures their content, allowing users to phrase their queries freely in natural language, rather than trying to match the exact keywords contained in a document.

Shaping Haystack 2.0

Mon, 14 Aug 2023 00:00:00 +0000

Since Haystack v1.15, we’ve been slowly introducing new components and features to Haystack in the background in preparation for Haystack 2.0 (or v2). After the work we’ve put into the new design of the Haystack API over the last few months, we’re at a point where we would love to start involving the Haystack community in our thought process and slowly gather your input and feedback. In this article, we would like to highlight where we are at for the design of the new Haystack API for v2, what we want to achieve with the new design, and what our current considerations are.

Question Answering in the Cockpit

Wed, 26 Jul 2023 00:00:00 +0000

Large language models (LLMs) are incredibly powerful, and we at deepset are stoked about their seemingly limitless potential. But while models like Llama 2 and GPT-4 continue to make waves, a related but less buzzworthy technology has consistently been delivering great value for a range of use cases.

Based on smaller, open-source Transformer models, extractive question answering (QA) is one of the most compelling knowledge management techniques to emerge from NLP in recent years. As a result, organizations are beginning to recognize the opportunities that extractive QA can bring to large knowledge bases.

Good Listener: How Memory Enables Conversational Agents

Fri, 07 Jul 2023 00:00:00 +0000

Memory is what turns a powerful LLM into an empathetic interlocutor who can remember what you’ve said before. But how does it work?

Today we want to lift the hood on how memory is implemented in Haystack. We’ll explain the differences between memory injection and memory as a tool and show you how to get around the context window’s length limitation by summarizing.

If you want to understand how memory works computationally or start building your own conversational AI interface with Haystack, this article is for you.

Share and Use Prompts with PromptHub

Thu, 29 Jun 2023 00:00:00 +0000

With the release of Haystack 1.18, we’ve also officially rolled out a new prompt sharing and collaboration service: PromptHub. This new product currently (day of release: 29 June 2023) hosts a set of prompts we at deepset have created, with instructions on how to use them with the PromptTemplate and PromptNode.

Prompts that are provided in PromptHub are all maintained in their own public repository on GitHub, deepset-ai/prompthub. Each prompt comes with a YAML file, housing the prompt itself, and a “prompt card” of the same name, which is a markdown file explaining the intended use case of the prompt and how to use it with a Haystack PromptNode.

What Is a Large Language Model (LLM)?

Fri, 23 Jun 2023 00:00:00 +0000

AI is here to stay. But while applications like autonomous driving and even image generation have only touched few people’s lives so far, generative AI in the form of high-quality chatbots has taken the world by storm. The informative, creative, and at times deceptively eloquent responses by the likes of ChatGPT are made possible by a novel technology known as large language models (LLMs).

In this article, we’ll talk about what LLMs are and how they’re produced, what kinds of LLMs exist, and whether it’s truly just their size that sets them apart from other language models. In the end, we’ll show you how you, too, can use LLMs in Haystack, our open source framework for NLP.

The Beginner's Guide to LLM Prompting

Thu, 15 Jun 2023 00:00:00 +0000

Large language models (LLMs) have made it possible for everyone to interact with an AI – not through code but using prompts in natural language. The fact that language now acts as an interface to complex models makes it necessary to investigate the prompts we use more closely.

When used correctly, generative models can produce highly valuable results for businesses. Getting your prompts right is therefore necessary to harness LLMs’ enormous potential, especially when you’re looking to incorporate it into your product. That is why an entire industry has formed around the topic of “prompt engineering.” In this post, we’ll explain our approach to this technique and share the dos and don’ts of prompting.

Run Haystack Pipelines in production with Ray Serve

Wed, 14 Jun 2023 00:00:00 +0000

Ray Serve is a library built on top of the Ray framework for building online inference APIs. Serve is designed to be framework-agnostic, and while its simple design lets you quickly integrate pretty much any Python logic you need to deploy, building up complex inference services is still possible and straightforward.

Haystack pipelines can be quite complex, but even the simpler ones consist of multiple components, which in turn might rely on different models and technologies - this aspect make them a good benchmark to test out Ray Serve’s capabilities.

Generative vs. Extractive Language Models

Mon, 22 May 2023 00:00:00 +0000

Generative language models like ChatGPT have taken the world by storm. Aside from their value in areas like programming and translation, generative models’ main selling point is their ability to come up with fluent, well-formed natural language responses. Like humans, these models seem to generate their answers out of thin air, and without looking up the answers in a factual database.

This sets them apart from their humbler siblings: extractive language models. Both extractive and generative models are based on the breakthrough Transformer architecture that ushered in a new generation of natural-language understanding (NLU) for machines.

How to Prevent Prompt Injections: An Incomplete Guide

Fri, 19 May 2023 00:00:00 +0000

ChatGPT is awesome, HuggingChat is awesome, Alpaca is awesome. However, if you want to use these models in an application, for example, for your customer support, you may encounter a new problem that you should be aware of: Prompt injections.

This guide showcases approaches to handling prompt injections. It also includes a brief overview of the first public prompt injection datasets and one of the first pre-trained prompt injection detection models available on Hugging Face that you can use to combat attacks against your system.

Scaling NLP indexing pipelines with KEDA and Haystack — Part 1: The Application

Mon, 01 May 2023 00:00:00 +0000

Large language models have been one of the most hyped technologies lately. By combining them with a vector database that acts as a long-term memory or document store, you can improve prompts with additional context. Applications that use this technique, such as ChatGPT Plugins and Google’s project Magi, are rapidly gaining popularity. For many business users, the ability to enhance a prompt with private or recent data is what makes the difference between a prototype and a production-ready NLP application.

Scaling NLP indexing pipelines with KEDA and Haystack — Part 2: The Deployment

Mon, 01 May 2023 00:00:00 +0000

In the first part of this article series, we discussed the power of retrieval-augmented generation. We also explored how to create a Python application that converts files into searchable documents with embeddings via Haystack pipelines. However, merely having a Python program that converts files into text snippets and embeddings on a single machine is not enough for a production-ready deployment.

In this part, we will explore how to deploy an indexing consumer to Kubernetes and how to autoscale it using KEDA. This will allow us to efficiently add text and embeddings to our vector database that can power a retrieval augmented LLM search engine like this.

New DocumentStore Integration for Haystack: QdrantDocumentStore

Tue, 18 Apr 2023 00:00:00 +0000

We’re thrilled to introduce the QdrantDocumentStore, the latest addition to the Haystack DocumentStore family by Qdrant. It’s an open source package that provides powerful capabilities such as efficient search, high-dimensional vector retrieval, and flexible launch options.

Let’s learn about DocumentStores and how to use QdrantDocumentStore with your pipelines!

Haystack DocumentStores

Haystack is an end-to-end NLP framework that provides a modular approach to building state-of-the-art generative AI, QA, and semantic knowledge base search systems. A core component of most modern NLP systems is a database that can efficiently store and retrieve vast amounts of text data. Vector databases are a great way of doing this. These databases can store vector representations of text while also implementing efficient ways to retrieve them at speed. To this end, Haystack provides a set of native DocumentStores that allow you to connect to and query your data that might be in Elasticsearch, Pinecone, Weaviate, or others. This way, you can store and maintain your data within one of these databases while simultaneously using them within your Haystack pipelines and applications.

Writing Professional Python Logs

Thu, 13 Apr 2023 00:00:00 +0000

If you are a Python developer who has been struggling with inconsistent and unhelpful logs, then this article is for you! In this post, we will discuss how structlog can help you write logs that not only assist you in your daily work but also enforce certain best practices to ensure consistency in logging throughout your project.

We will also explore common logging challenges and how structlog can help overcome them. So whether you are a beginner or an experienced developer, keep reading to learn how structlog can revolutionize your logging process!

Introducing Agents in Haystack: Make LLMs resolve complex tasks

Thu, 30 Mar 2023 00:00:00 +0000

Wouldn’t it be cool if you could enter a query and an LLM makes decisions on-the-fly about what resources it needs to resolve the query? For example, that it needs to search the web and retrieve relevant resources. Or that it needs to search through your company files first. That’s now possible with Agents!

With the release of Haystack 1.15, we’re officially introducing the Agent to the Haystack ecosystem. The implementation of Agents is inspired by two papers: the MRKL Systems paper (pronounced ‘miracle’ 😉) and the ReAct paper. If you like reading papers, I highly recommend these two. Here, I’ll explain how we’re introducing this functionality to Haystack.

Announcing the Haystack Blog

Fri, 03 Mar 2023 00:00:00 +0000

We are thrilled to announce the launch of our new blog, a space to learn and share! The Haystack community has been growing at an incredible speed, reaching over 7k stars on GitHub and 900+ members on Discord, and we’re always looking for ways to make our community experience even better. Haystack blog is another step in that direction. 🚀

You might already be familiar with our deepset blog where we share tons of great articles. While we will continue to publish new content there too, on this new blog you will find more content targeted to developers building with Haystack specifically. It will be easier and faster for the Haystack developer to find the NLP content that is most relevant to them! 🏎️

Build a Search Engine with GPT-3

Tue, 31 Jan 2023 00:00:00 +0000

If you’ve been online lately, then you’ve likely seen the excitement about OpenAI’s newest language model, ChatGPT. ChatGPT is astonishingly good at many things, including debugging code and rewriting text in whatever style you ask it. As an offshoot of GPT-3.5, a large language model (LLM) with billions of parameters, ChatGPT owes its impressive amount of knowledge to the fact that it’s seen a large portion of the internet during training — in the form of the Common Crawl corpus and other data.

The Definitive Guide to BERT Models

Mon, 16 Jan 2023 00:00:00 +0000

Anyone who has studied natural language processing (NLP) can tell you that the state of the art moves exceptionally fast. Big players like Google, Facebook, or OpenAI employ large teams of experts to come up with new solutions that bring computers ever closer to a seemingly human-like understanding of language. This results in model architectures and other approaches quickly becoming obsolete, and what was considered cutting-edge technology six months ago may almost seem outdated today. Nevertheless, some models make such an impact that they become foundational knowledge even as they are eclipsed by their successors.

How to Build a Semantic Search Engine in Python

Wed, 23 Nov 2022 00:00:00 +0000

Semantic search is the task of retrieving documents from a collection of documents (also known as a ‘corpus’) in response to a query asked in natural language. Powered by the latest Transformer language models, semantic search allows you to access the best matches from your document collection within seconds, and on the basis of meaning rather than keyword matches. As well as being helpful in its own right, semantic search also forms the basis for many complex tasks, like question answering or text summarization.

When and How to Train Your Own Language Model

Wed, 03 Aug 2022 00:00:00 +0000

Many people, when considering whether to incorporate natural language processing functionality into their product, shy away from the perceived complexity of the task. Modern-day NLP operates with huge language models that learn from large amounts of data.

What many beginners don’t know, however, is that the vast majority of use cases don’t require training a new language model from scratch. There are already tens of thousands of pre-trained models freely available online, which can be used out of the box.

What is a Language Model?

Wed, 20 Jul 2022 00:00:00 +0000

Our aim at deepset is that everyone, no matter their level of technical background, can harness the power of modern natural language processing (NLP) and language models for their own use case. Haystack, our open-source framework, makes this a reality.

When we talk to our users, we encounter common sources of confusion about NLP and machine learning. Therefore, in the upcoming blog posts, we want to explain some basic NLP concepts in understandable language. First up: language models.

What Is Text Vectorization? Everything You Need to Know

Fri, 03 Dec 2021 00:00:00 +0000

For as long as we have had computers, there has been the question of how to represent data in a way that machines can work with. In natural language processing (NLP), we often talk about text vectorization — representing words, sentences, or even larger units of text as vectors (or “vector embeddings”). Other data types, like images, sound, and videos, may be encoded as vectors as well. But what exactly are those vectors, and how can you use them in your own applications?

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Adanos provides structured market sentiment for stocks and crypto. The AdanosMarketSentiment component makes signals from Reddit, X / FinTwit, financial news, and Polymarket available to Haystack pipelines and agents without adding trading or portfolio logic.

The community-maintained integration supports sentiment lookups, trending assets, aggregate market sentiment, asset comparisons, search, and dataset statistics. Stock requests can use Reddit, X, news, or Polymarket; crypto requests use Reddit.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AIMLAPIChatGenerator lets you call any of the models that AIMLAPI serves behind an OpenAI-compatible /chat/completions endpoint. AIMLAPI routes the request to the requested provider while maintaining the familiar OpenAI payload schema, so you can reuse existing Haystack pipelines or agents with minimal changes.

AIMLAPI extends the base OpenAI integration with:

Direct AIMLAPI routing – requests are sent to https://api.aimlapi.com/v1 and can target any model AIMLAPI exposes by passing the model name.
Tool calling support – pass Haystack Tool objects to the generator to enable function calling workflows.
Streaming callbacks – supply streaming_callback to receive tokens as they are generated.
Flexible extras – forward provider-specific parameters by using generation_kwargs and extra_body.

To follow along with the example below, set the AIMLAPI_API_KEY environment variable to your API token.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

AlloyDB is a fully managed, PostgreSQL-compatible database service on Google Cloud, optimised for demanding transactional and analytical workloads.

This integration provides a Haystack DocumentStore backed by AlloyDB with the pgvector extension, enabling both dense vector similarity search and full-text keyword search.

Connections are established through the AlloyDB Python Connector, which handles IAM-based authentication and TLS encryption without requiring manual firewall rules or IP allowlisting.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AlphaAI is a financial-news platform built for AI agents: every article is enriched at ingest with per-ticker impact analysis, a category, and a 1-10 relevance score, and SEC Form 4 insider filings become structured events about 6 minutes after they hit EDGAR.

This integration provides:

AlphaAINewsFetcher: fetches the scored news feed as Haystack Document objects, with ticker / category / relevance filters and optional same-story collapsing.
AlphaAIInsiderNewsFetcher: fetches SEC Form 4 insider events with a structured meta["insider"] block (side, shares, average price, total value, who traded, 10b5-1 flag).

An API key is required — the free tier (20 requests/min, 100/day, no card) is enough to try it: alphai.io/developers. The components read the key from the ALPHAI_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Amazon Bedrock is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available for your use through a unified API. You can choose from various foundation models to find the one best suited for your use case. More information can be found on the Amazon Bedrock documentation page.

Installation

Install the Amazon Bedrock integration:

pip install amazon-bedrock-haystack

Usage

Once installed, you will have access to AmazonBedrockChatGenerator and AmazonBedrockGenerator components that support generative language models on Amazon Bedrock. You will also have access to the AmazonBedrockTextEmbedder and AmazonBedrockDocumentEmbedder, which can be used to compute embeddings. The integration also includes S3Downloader that allows downloading files from AWS S3 buckets to the local filesystem.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Amazon Sagemaker is a comprehensive, fully managed machine learning service that allows data scientists and developers to build, train, and deploy ML models efficiently. More information can be found on the documentation page.

Installation

Install the Amazon Sagemaker integration:

pip install amazon-sagemaker-haystack

Usage

Once installed, you will have access to a SagemakerGenerator that supports models from various providers. To know more about which models are supported, check out Sagemaker’s documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AmazonTextractConverter provides an integration of Amazon Textract with Haystack.

This component uses Amazon Textract’s synchronous API to convert images and single-page PDFs into Haystack Document objects using OCR. It supports plain text extraction, structural analysis for tables and forms, and natural-language queries on documents.

Supported file formats: JPEG, PNG, TIFF, BMP, and single-page PDF (up to 10 MB).

Key features:

Plain text extraction with DetectDocumentText
Table, form, signature, and layout detection with AnalyzeDocument
Natural-language queries to extract specific answers from documents
Access to the raw Textract response for downstream processing

Installation

Install the Amazon Textract integration:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration supports Anthropic Claude models such as Claude Haiku 3.5, Claude Sonnet 3.7, and Claude Sonnet 4.5 through Anthropic’s inference infrastructure. For a complete list of available models, check out the Anthropic Claude documentation.

You can use Anthropic models with AnthropicGenerator and AnthropicChatGenerator.

Installation

pip install anthropic-haystack

Usage

Based on your use case, you can choose between AnthropicGenerator or AnthropicChatGenerator to work with Anthropic models. To learn more about the difference, visit the Generators vs Chat Generators guide.
Before using, make sure to set the ANTHROPIC_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Apify is a web scraping and data extraction platform. It helps automate web tasks and extract content from e-commerce websites, social media (Facebook, Instagram, TikTok), search engines, online maps, and more. Apify provides more than two thousand ready-made cloud solutions called Actors.

Installation

Install the Apify-haystack integration:

pip install apify-haystack

Usage

Once installed, you will have access to more than two thousand ready-made apps called Actors at Apify Store

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

ArangoDB is an open-source, multi-model database that combines documents, graphs, and key/values with native vector search. This integration lets you use ArangoDB as a Document Store in Haystack and retrieve documents with vector similarity search, which makes it a good fit for RAG and GraphRAG pipelines.

The integration provides two components:

ArangoDocumentStore: a Document Store that stores documents (including their embeddings) in an ArangoDB collection and implements the DocumentStore protocol.
ArangoEmbeddingRetriever: a retriever that fetches the most relevant documents from an ArangoDocumentStore using vector similarity on embeddings.

Installation

Vector search requires ArangoDB 3.12 or later with the vector index enabled. You can quickly start a local instance with Docker:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

An integration of ArcadeDB with Haystack by ArcadeData.

ArcadeDB is a multi-model database that combines document storage, HNSW vector search, and SQL-based metadata filtering:

Document storage — vertex-based records with flexible MAP metadata
HNSW vector search — native approximate nearest neighbor index via vectorNeighbors() (cosine, euclidean, dot product)
SQL filtering — full SQL WHERE clauses on metadata fields
No special drivers — pure HTTP/JSON API, no binary protocol or custom driver required

The library provides an ArcadeDBDocumentStore that implements the Haystack DocumentStore protocol, plus pipeline-ready retriever components:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Arize Phoenix is Arize’s open-source platform that offers developers the quickest way to troubleshoot, evaluate, and experiment with LLM applications.

For a detailed integration guide, see the documentation for Phoenix + Haystack

Installation

pip install openinference-instrumentation-haystack haystack-ai arize-phoenix

Usage

To trace any Haystack pipeline with Phoenix, simply initialize OpenTelemetry and the HaystackInstrumentor. Haystack pipelines that run within the same environment send traces to Phoenix.

First, start a Phoenix instance to send traces to.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Arize is AI Observability and Evaluation platform designed to help you troubleshoot, evaluate, and experiment on LLM and ML applications. Developers use Arize to get applications working quickly, evaluate performance, detect and prevent production issues, and curate datasets.

Documentation for Arize AI + Haystack

Installation

pip install openinference-instrumentation-haystack haystack-ai arize-otel opentelemetry-sdk opentelemetry-exporter-otlp

Usage

To trace any Haystack pipeline with Arize, simply initialize OpenTelemetry and the HaystackInstrumentor. Haystack pipelines that run within the same environment send traces to Arize.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AI agents running in production need more than observability. When a pipeline makes a decision, calls a tool, or generates output, you need tamper-evident proof of what happened, not just a log entry someone could edit later. This matters for regulatory compliance (EU AI Act Article 12 requires automatic, verifiable event recording for high-risk systems), incident investigation (reconstructing exactly what an agent did and why), and accountability across teams that share pipelines.

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

You can use AssemblyAI trancriptions in your Haystack pipelines with the AssemblyAITranscriber.

With this integration, you can perform speech recognition, speaker diarization and summarization.

More info about AssemblyAI:

Installation

pip install assemblyai-haystack

Usage

The AssemblyAITranscriber allows to perform some speech-to-text processes using the AssemblyAI API and loads the transcribed text into documents. To use this component, you should pass your ASSEMBLYAI_API_KEY as an argument.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

DataStax Astra DB is a serverless vector database built on Apache Cassandra, and it supports vector-based search and auto-scaling. You can deploy it on AWS, GCP, or Azure and easily expand to one or more regions within those clouds for multi-region availability, low latency data access, data sovereignty, and to avoid cloud vendor lock-in. For more information, see the DataStax documentation.

This integration allows you to use AstraDB for document storage and retrieval in your Haystack pipelines. This page provides instructions on how to initialize an AstraDB instance and connect with Haystack.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AzureAIDocumentStore supports an integration of Azure AI Search which is an enterprise-ready search and retrieval system with Haystack by deepset.

This integration allows using search indexes in Azure AI Search as a document store to build RAG-based applications on Azure, with native LLM integrations. To retrieve data from the document store, the integration supports three types of retrieval techniques:

Embedding Retrieval: For vector-based searches.
BM25 Retrieval: Keyword retrieval utilizing the BM25 algorithm.
Hybrid Retrieval: A combination of vector and BM25 retrieval methods.

Installation

Install the Azure AI Search integration:

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

Azure Cosmos DB is a fully managed NoSQL, relational, and vector database for modern app development. It offers single-digit millisecond response times, automatic and instant scalability, and guaranteed speed at any scale. It is the database that ChatGPT relies on to dynamically scale with high reliability and low maintenance. Haystack supports MongoDB and PostgreSQL clusters running on Azure Cosmos DB.

Azure Cosmos DB for MongoDB makes it easy to use Azure Cosmos DB as if it were a MongoDB database. You can use your existing MongoDB skills and continue to use your favorite MongoDB drivers, SDKs, and tools by pointing your application to the connection string for your account using the API for MongoDB. Learn more in the Azure Cosmos DB for MongoDB documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AzureDocumentIntelligenceConverter provides an integration of Azure Document Intelligence (formerly Form Recognizer) with Haystack by deepset.

This component uses Azure’s Document Intelligence service to convert various file formats into Haystack Documents with markdown content. It supports advanced document analysis including layout detection, table extraction, and structured content recognition.

Supported file formats: PDF, JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, HTML.

Key features:

Markdown output with preserved structure (headings, tables, lists)
Inline table integration (tables rendered as markdown tables)
Improved layout analysis and reading order
Support for section headings
Multiple model options for different use cases

Installation

Install the Azure Document Intelligence integration:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

AzureOCRDocumentConverter converts files to Haystack Documents using Azure’s Document Intelligence service through the azure-ai-formrecognizer SDK.

Supported file formats: PDF, JPEG, PNG, BMP, TIFF, DOCX, XLSX, PPTX, HTML.

Unlike the AzureDocumentIntelligenceConverter (which produces Markdown), this component extracts tables as separate Document objects that preserve their two-dimensional (CSV) structure, and returns the remaining text with page breaks (\f) so it can be split per page by downstream preprocessors.

You need an active Azure account and a Document Intelligence or Cognitive Services resource. Follow the Azure setup guide to create your resource.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Azure OpenAI Service provides REST API access to OpenAI’s powerful language models including the GPT-4, GPT-4 Turbo with Vision, GPT-3.5-Turbo, and Embeddings model series. To get access to Azure OpenAI endpoints, visit Azure OpenAI Service REST API reference.

Installation

Install Haystack:

pip install haystack-ai

Usage

To work with Azure components, you will need an Azure OpenAI API key, an Azure Active Directory Token as well as an Azure OpenAI Endpoint.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Brave Search is an independent search engine with its own web index. Unlike most search APIs, it does not rely on Google or Bing, making it a great choice for privacy-conscious applications.

This integration provides:

BraveWebSearch: Searches the web using the Brave Search API and returns results as Haystack Document objects along with source URLs.

You need a Brave Search API key to use this integration. You can get one at brave.com/search/api.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Bright Data is the world’s leading web data platform, providing enterprise-grade web scraping and data collection solutions. The Bright Data Haystack integration provides three powerful components for extracting and accessing web data:

Key Features:

Web Scraper: Extract structured data from 45+ supported websites including Amazon, LinkedIn, Instagram, Facebook, TikTok, YouTube, and more
SERP API: Get search engine results from Google, Bing, Yahoo with geo-targeting and language customization
Web Unlocker: Access geo-restricted and bot-protected websites, bypass CAPTCHAs and anti-bot measures

Use Cases:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Burr is an open-source Python framework for building applications that make decisions (chatbots, agents, simulations, etc.). Burr UI is its open-source, free, and local-first companion app for monitoring, debugging, annotating, and more.

This extension lets you use your Haystack components (native, integrations, and custom) to create Burr agents. Burr offers flexibility to manage the state of your agent (i.e., memory) and define conditional logic to select its next action.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cerebras is the go-to platform for fast and effortless AI training and inference.

Usage

Cerebras API is OpenAI compatible, making it easy to use in Haystack via OpenAI Generators.

Using `Generator`

Here’s an example of using llama3.1-8b served via Cerebras to perform question answering on a web page. You need to set the environment variable CEREBRAS_API_KEY and choose a compatible model.

from haystack import Pipeline
from haystack.utils import Secret
from haystack.components.fetchers import LinkContentFetcher
from haystack.components.converters import HTMLToDocument
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator

fetcher = LinkContentFetcher()
converter = HTMLToDocument()
prompt_template = """
According to the contents of this website:
{% for document in documents %}
  {{document.content}}
{% endfor %}
Answer the given question: {{query}}
Answer:
"""
prompt_builder = PromptBuilder(template=prompt_template)
llm = OpenAIGenerator(
    api_key=Secret.from_env_var("CEREBRAS_API_KEY"),
    api_base_url="https://api.cerebras.ai/v1",
    model="llama3.1-8b"
)
pipeline = Pipeline()
pipeline.add_component("fetcher", fetcher)
pipeline.add_component("converter", converter)
pipeline.add_component("prompt", prompt_builder)
pipeline.add_component("llm", llm)

pipeline.connect("fetcher.streams", "converter.sources")
pipeline.connect("converter.documents", "prompt.documents")
pipeline.connect("prompt.prompt", "llm.prompt")

result = pipeline.run({"fetcher": {"urls": ["https://cerebras.ai/inference"]},
              "prompt": {"query": "Why should I use Cerebras for serving LLMs?"}})

print(result["llm"]["replies"][0])

Using `ChatGenerator`

See an example of engaging in a multi-turn conversation with llama3.1-8b. You need to set the environment variable CEREBRAS_API_KEY and choose a compatible model.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Chainlit is an open-source Python package for building production-ready Conversational AI. By exposing your Haystack app (standalone agent or pipeline) through Hayhooks as OpenAI-compatible endpoints, you can run the Chainlit chat UI inside your Hayhooks server, giving you a zero-configuration frontend to interact with your deployed pipelines without a separate client.

For full details, see the Hayhooks Chainlit integration guide.

Installation

Install Hayhooks with the chainlit extra:

pip install "hayhooks[chainlit]"

Usage

Hayhooks Quick Start

The simplest way to enable the Chainlit UI is via the --with-chainlit flag:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Chonkie is a fast, lightweight chunking library designed for RAG applications. This integration provides four Haystack document splitter components backed by Chonkie’s chunkers:

Component	Chunking strategy
`ChonkieTokenDocumentSplitter`	Fixed-size token-based chunks with configurable overlap
`ChonkieSentenceDocumentSplitter`	Chunks that respect sentence boundaries
`ChonkieRecursiveDocumentSplitter`	Hierarchical recursive splitting using a rule set
`ChonkieSemanticDocumentSplitter`	Embedding-based splitting at semantic topic boundaries

All components accept a list[Document] and return a list[Document]. Each output document carries source_id, page_number, split_id, split_idx_start, split_idx_end, and token_count in its metadata.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Chroma Document Store for Haystack

Installation

Use pip to install Chroma:

pip install chroma-haystack

Usage

Once installed, initialize your Chroma database to use it with Haystack:

from haystack_integrations.document_stores.chroma import ChromaDocumentStore

# Chroma is used in-memory so we use the same instances in the two pipelines below
document_store = ChromaDocumentStore()

Writing Documents to ChromaDocumentStore

To write documents to ChromaDocumentStore, create an indexing pipeline.

from haystack.components.converters import TextFileToDocument
from haystack.components.writers import DocumentWriter

indexing = Pipeline()
indexing.add_component("converter", TextFileToDocument())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("converter", "writer")
indexing.run({"converter": {"sources": file_paths}})

Examples

You can find a code example showing how to use the Document Store and the Retriever under the example/ folder of this repo or follow 📓 Chroma Indexing and RAG Examples.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Cognee provides open-source, knowledge-graph-backed memory for AI agents and assistants. It helps Haystack applications store facts, conversation history, and contextual knowledge as a semantic graph, then retrieve the most relevant memories using graph-completion or other search strategies.

The cognee-haystack package is part of Haystack Core Integrations and provides:

CogneeMemoryStore: A persistent memory store backed by Cognee’s knowledge graph API.
CogneeRetriever: A pipeline component for retrieving memories from Cognee as system ChatMessage objects.
CogneeWriter: A pipeline component for writing ChatMessage memories to Cognee.

Cognee supports two memory tiers:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

You can use Cohere Models in your Haystack pipelines with the Generators and Embedders.

Installation

pip install cohere-haystack

Usage

You can use Cohere models in various ways:

Embedding Models

You can leverage /embed models from Cohere through three components: CohereTextEmbedder, CohereDocumentEmbedder and CohereDocumentImageEmbedder. These components support the Embed series of models.

To create semantic embeddings for textual documents, use CohereDocumentEmbedder in your indexing pipeline. To create semantic embeddings for image-based documents, use CohereDocumentImageEmbedder in your indexing pipeline. For generating embeddings for queries, use CohereTextEmbedder. Once you’ve selected the suitable component for your specific use case, initialize the component with the model name. By default, the Cohere API key with be automatically read from either the COHERE_API_KEY environment variable or the CO_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

CometAPIChatGenerator lets you call any LLMs available on Comet API, including:

OpenAI variants such as gpt-5
Anthropic’s claude-4.5-haiku
Community-hosted open-source models (Llama 2, Mixtral, etc.)

For more information on models available via the Comet API API, see the Comet API docs.

In order to follow along with this guide, you’ll need a Comet API key. Add it as an environment variable, COMET_API_KEY.

Installation

pip install cometapi-haystack

Usage

You can use CometAPIChatGenerator as standalone, within a pipeline or with the Agent component.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Context.ai is an evaluations and analytics tool for products powered by LLMs.

With Context.ai, you can understand how your users are interacting with natural language interfaces. This helps you know where your customers are having great experiences, but also proactively detect potential areas of improvement. You can test the performance impact of changes before you ship them to production with evaluations, and can identify where inappropriate conversations taking place.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Breaking Changes in Version Couchbase 2.0.0

Important Note:
In version 2.0.0, the following component names have been changed:

CouchbaseDocumentStore is now CouchbaseSearchDocumentStore

CouchbaseEmbeddingRetriever is now CouchbaseSearchEmbeddingRetriever

Please update your code accordingly if upgrading from an earlier version.

Overview

An integration of Couchbase NoSQL database with Haystack by deepset. Couchbase supports three types of vector indexes for AI applications, and this library provides document stores for two of them:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration lets you use Datadog to trace and monitor your Haystack pipelines and agents. It relies on Datadog’s tracing library ddtrace and provides a DatadogConnector component that, once added to your pipeline, sends Haystack traces to Datadog.

Installation

pip install datadog-haystack

Usage

Add the DatadogConnector to your pipeline without connecting it to any other component. It enables Datadog tracing for all pipeline operations.

You also need to set the HAYSTACK_CONTENT_TRACING_ENABLED environment variable to true to trace the content (inputs and outputs) of the pipeline components.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

ddgs (“Dux Distributed Global Search”) is a free metasearch library that aggregates results from multiple backends — DuckDuckGo, Google, Bing, Brave, Yahoo, Yandex, and more.

This integration provides:

DDGSWebSearch: Searches the web through ddgs and returns results as Haystack Document objects along with source URLs.

No API key is required.

Installation

pip install ddgs-haystack

Usage

DDGSWebSearch

DDGSWebSearch queries multiple search backends through ddgs and returns results as Haystack Document objects containing the content snippets and metadata (title, URL). Source URLs are also returned separately.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

DeepEval (by Confident AI) is an open source framework for model-based evaluation to evaluate your LLM applications by quantifying their performance on aspects such as faithfulness, answer relevancy, contextual recall etc. More information can be found on the documentation page.

Installation

Install the DeepEval integration:

pip install deepeval-haystack

Usage

Once installed, you will have access to a DeepEvalEvaluator that supports a variety of model-based evaluation metrics:

Answer Relevancy
Faithfulness
Contextual Precision
Contextual Recall
Contextual Relevance

In addition to evaluation scores, DeepEval’s evaluators offer additional reasoning for each evaluation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

DeepL is a powerful translation services provider, offering high-quality translations in multiple languages. This integration allows you to use DeepL’s translation services with Haystack.

Installation

pip install deepl-haystack

Usage

Components

The DeepL Haystack integration introduces two components that can be used to obtain translations using the DeepL API.

The DeepLTextTranslator to translate plain text (Python strings).
The DeepLDocumentTranslator to translate Haystack Document objects.

API Key

To use the DeepL Haystack integration, you’ll need to provide a DeepL API key. You can get one by signing up at the DeepL API website.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Dewey is a managed document intelligence backend for AI applications. Upload PDFs, Word docs, and other files — Dewey handles conversion, section extraction, chunking, embedding, and hybrid semantic + BM25 retrieval automatically.

This integration provides three Haystack 2.0 components:

DeweyDocumentStore — implements the Haystack DocumentStore protocol, backed by a Dewey collection
DeweyRetriever — a @component that runs hybrid search against a collection and returns ranked Document objects
DeweyResearchComponent — a @component that runs Dewey’s full agentic research loop (multi-step search, synthesis, citations) and returns a grounded Markdown answer

Installation

pip install dewey-haystack

Requires a free Dewey account at meetdewey.com. Set your API key:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Docling Serve hosts Docling as a scalable HTTP server, supporting PDFs, Office documents, HTML, and many other formats. All document parsing happens on the remote server, with no local ML dependencies.

Installation

pip install docling-serve-haystack

Start a Docling Serve instance locally (requires Docker):

docker run -p 5001:5001 ghcr.io/docling-project/docling-serve-cpu:latest

Usage

Components

DoclingServeConverter converts documents by sending them to a Docling Serve HTTP server. Local files and ByteStream objects are uploaded via the /v1/convert/file endpoint. URL strings are sent to /v1/convert/source.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Docling locally parses PDF, DOCX, HTML, and other document formats into a rich standardized representation (incl. layout, tables etc.), which it can then export to Markdown, JSON, and others.

Check out the Docling docs for more details.

This integration introduces Docling support, enabling Haystack users to:

use various document types in LLM applications with ease and speed, and
leverage Docling’s rich format for advanced, document-native grounding.

Installation

pip install docling-haystack

Usage

Components

This integration introduces DoclingConverter, a component which reads document file paths (local or URL) and outputs Haystack Document objects.

Mon, 01 Jan 0001 00:00:00 +0000

Implements a component of the kind WebSearch, but through the freely available DuckDuckGo API.

Overview

DuckduckgoApiWebSearch performs web searches using the DuckDuckGo search engine.

This repository provides a Python module similar to SearchApiWebSearch and SerperDevWebSearch, but utilizes the free DuckDuckGo API.

When you pass a query to DuckduckgoWebSearch, it returns a list of URLs that are most relevant to your search. The results are based on page snippets (the brief text displayed beneath the page titles in search results) rather than the content of the entire page.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

E2B provides secure, isolated cloud sandboxes that let LLM-powered agents execute arbitrary code and shell commands without touching the host machine. The e2b-haystack integration wraps the E2B SDK as a set of Haystack Tools that an Agent can invoke to run bash commands and manage files inside a shared sandbox.

All tools provided by the integration operate on the same live E2BSandbox, so files written by one tool call are immediately available to the next — the agent can, for example, write a Python script with write_file, execute it with run_bash_command, and read the result back with read_file.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Eden AI is a unified, OpenAI-compatible API that gives access to 500+ AI models from many providers (OpenAI, Anthropic, Mistral, Google, Cohere, and more) through a single API key, with built-in provider fallback and EU data residency. This makes it a convenient, sovereignty-friendly gateway for building LLM and RAG applications with Haystack.

Models are selected using Eden AI’s provider/model naming convention, for example openai/gpt-4o-mini, anthropic/claude-sonnet-4-5, or mistral/mistral-large-latest. See the Eden AI models catalog for the full list.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The ElasticsearchDocumentStore is maintained in haystack-core-integrations repo. It allows you to use Elasticsearch as data storage for your Haystack pipelines.

For a details on available methods, visit the API Reference

Installation

To run an Elasticsearch instance locally, first follow the installation and start up guides.

pip install elasticsearch-haystack

Usage

Once installed, you can start using your Elasticsearch database with Haystack by initializing it:

from haystack_integrations.document_stores.elasticsearch import ElasticsearchDocumentStore

document_store = ElasticsearchDocumentStore(hosts = "http://localhost:9200")

Writing Documents to ElasticsearchDocumentStore

To write documents to your ElasticsearchDocumentStore, create an indexing pipeline with a DocumentWriter, or use the write_documents() function. For this step, you can use the available TextFileToDocument and DocumentSplitter, as well as other Integrations that might help you fetch data from other resources.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This repository contains an integration of ElevenLabs’ Text-to-Speech API with Haystack pipelines. This package allows you to convert text to speech using ElevenLabs’ API and optionally save the generated audio to AWS S3.

Installation

pip install elevenlabs_haystack

Usage

ElevenLabs API Key

To access the ElevenLabs API, you need to create an account and obtain an API key.

Go to the ElevenLabs website and sign up for an account.
Once logged in, navigate to the Profile section.
In the API section, generate a new API key.
Copy the API key.

AWS Credentials

To store generated audio files on AWS S3, you need AWS credentials (Access Key ID, Secret Access Key) and specify a region.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

EmpirioLabs is a multi-model API platform that hosts open and proprietary models (Qwen, DeepSeek, GLM, Kimi, MiniMax, Gemma, and more) behind one OpenAI-compatible API with pay-as-you-go pricing.

To start using EmpirioLabs, create an API key in the EmpirioLabs dashboard. The full model catalog with per-model context windows and pricing is at empiriolabs.ai/models.

Usage

The EmpirioLabs API is OpenAI compatible, making it easy to use in Haystack via the OpenAI Generators and Embedders.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Exa integration for Haystack provides components to search the web, fetch content, get AI-powered answers, and conduct deep research using Exa’s API.

Installation

pip install exa-haystack

Components

ExaWebSearch: AI-powered web search with multiple speed/quality modes (auto, instant, fast, deep, deep-reasoning, deep-max, neural)
ExaFindSimilar: Find pages similar to a URL
ExaContents: Fetch full content for URLs with freshness control
ExaAnswer: Get AI-powered answers with citations and optional structured output
ExaStreamAnswer: Streaming answers with SSE and optional structured output
ExaResearch: Deep research with automatic source gathering

Usage

Web Search

from haystack_integrations.components.websearch.exa import ExaWebSearch

search = ExaWebSearch(num_results=5, type="auto", text=True)
results = search.run(query="latest AI developments")

for doc in results["documents"]:
    print(doc.meta["title"], doc.meta["url"])

Use type="instant" for sub-150ms searches, type="deep" or type="deep-reasoning" for higher-quality results, or type="auto" (default) to let Exa choose.

Mon, 01 Jan 0001 00:00:00 +0000

The integration provides FAISSDocumentStore, which uses FAISS (Facebook AI Similarity Search) for vector search and a simple JSON file for metadata storage. It is suitable for small to medium-sized datasets where simplicity is preferred over scalability, and supports optional persistence by saving the FAISS index to a .faiss file and documents to a .json file. Use FAISSEmbeddingRetriever for semantic retrieval in your pipelines.

Installation

Install the package with pip:

pip install faiss-haystack

For GPU-accelerated FAISS, install faiss-gpu separately and use it in place of the default faiss-cpu dependency where applicable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

An integration of FalkorDB with Haystack by deepset.

FalkorDB is a high-performance graph database optimized for GraphRAG workloads. It stores documents as graph nodes and supports native vector search — no APOC is required. All bulk writes use UNWIND + MERGE for safe, idiomatic OpenCypher upserts.

The library provides a FalkorDBDocumentStore that implements the Haystack DocumentStore protocol, plus two pipeline-ready retriever components:

FalkorDBDocumentStore — stores Documents as labeled graph nodes in a named FalkorDB graph, with meta fields stored flat alongside id and content. Embeddings are indexed using FalkorDB’s native vector index.
FalkorDBEmbeddingRetriever — a retriever component that queries the native vector index to find Documents by dense similarity, with support for metadata filtering.
FalkorDBCypherRetriever — a power-user retriever for executing arbitrary OpenCypher queries, enabling graph traversal and multi-hop queries in GraphRAG pipelines.

                                   +-----------------------------+
                                   |      FalkorDB Database      |
                                   +-----------------------------+
                                   |                             |
                                   |      +----------------+     |
                                   |      |    Document    |     |
                write_documents    |      +----------------+     |
          +------------------------+----->|   properties   |     |
          |                        |      |                |     |
+---------+----------+             |      |   embedding    |     |
|                    |             |      +--------+-------+     |
| FalkorDBDocument   |             |               |             |
|       Store        |             |               |index/query  |
+---------+----------+             |               |             |
          |                        |     +---------+---------+   |
          |                        |     | Native Vector Idx |   |
          +----------------------->|     |                   |   |
              _embedding_retrieval |     |  (vecf32 index)   |   |
                                   |     +-------------------+   |
                                   |                             |
                                   +-----------------------------+

In the above diagram:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

FastEmbed is a lightweight, fast, Python library built for embedding generation and document ranking.

Light and fast: quantized model weights; ONNX Runtime for inference via Optimum.
Performant embedding models: list of supported models - including multilingual models.
Support for sparse embedding models.
Good integration with Qdrant document store and retrievers.

Installation

pip install fastembed-haystack

Usage

Components

The fastembed-haystack integrations provides the following components:

Embedders:
- FastembedTextEmbedder: creates a dense embedding for text (used in query/RAG pipelines).
- FastembedDocumentEmbedder: enriches documents with dense embeddings (used in indexing pipelines).
- FastembedSparseTextEmbedder: creates a sparse embedding for text (used in query/RAG pipelines).
- FastembedSparseDocumentEmbedder: enriches documents with sparse embeddings (used in indexing pipelines).
Rankers:
- FastembedRanker: ranks documents based on a query using cross-encoder models (used in query/RAG pipelines after the retrieval).
- FastembedLateInteractionRanker: ranks documents using ColBERT late-interaction (MaxSim) scoring (used in query/RAG pipelines after the retrieval).

Example with dense embeddings

from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.embedders.fastembed import FastembedDocumentEmbedder, FastembedTextEmbedder

document_store = InMemoryDocumentStore(embedding_similarity_function="cosine")

documents = [
    Document(content="My name is Wolfgang and I live in Berlin"),
    Document(content="I saw a black horse running"),
    Document(content="Germany has many big cities"),
    Document(content="fastembed is supported by and maintained by Qdrant."),
]

document_embedder = FastembedDocumentEmbedder()
documents_with_embeddings = document_embedder.run(documents)["documents"]
document_store.write_documents(documents_with_embeddings)

query_pipeline = Pipeline()
query_pipeline.add_component("text_embedder", FastembedTextEmbedder())
query_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
query_pipeline.connect("text_embedder.embedding", "retriever.query_embedding")

query = "Who supports fastembed?"

result = query_pipeline.run({"text_embedder": {"text": query}})

For a more detailed example, see this notebook.

Mon, 01 Jan 0001 00:00:00 +0000

fastRAG is a research framework for efficient and optimized retrieval augmented generative pipelines, incorporating state-of-the-art LLMs and Information Retrieval. fastRAG is designed to empower researchers and developers with a comprehensive tool-set for advancing retrieval augmented generation.

Comments, suggestions, issues and pull-requests are welcomed! ❤️

📣 Updates

2024-05: fastRAG V3 is Haystack 2.0 compatible 🔥
2023-12: Gaudi2 and ONNX runtime support; Optimized Embedding models; Multi-modality and Chat demos; REPLUG text generation.
2023-06: ColBERT index modification: adding/removing documents.
2023-05: RAG with LLM and dynamic prompt synthesis example.
2023-04: Qdrant DocumentStore support.

Key Features

Optimized RAG: Build RAG pipelines with SOTA efficient components for greater compute efficiency.
Optimized for Intel Hardware: Leverage Intel extensions for PyTorch (IPEX), 🤗 Optimum Intel and 🤗 Optimum-Habana for running as optimal as possible on Intel® Xeon® Processors and Intel® Gaudi® AI accelerators.
Customizable: fastRAG is built using Haystack and HuggingFace. All of fastRAG’s components are 100% Haystack compatible.

🚀 Components

For a brief overview of the various unique components in fastRAG refer to the Components Overview page.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Featherless AI is a serverless AI inference platform. Our goal is to make all AI models available for serverless inference and we’ve started with large language models (e.g. Qwen, Llama, Mistral, DeepSeek, RWKV). We provide inference via API to a continually expanding library of open-weight models, including the most popular models for role-playing, creative writing, coding assistance, and more.

To start using Featherless, sign up for an API key here.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Firecrawl turns websites into LLM-ready data. It handles JavaScript rendering, anti-bot bypassing, and outputs clean Markdown.

This integration provides two components:

FirecrawlCrawler: Crawls one or more URLs and follows links to discover subpages, returning extracted content as Haystack Document objects.
FirecrawlWebSearch: Searches the web using a query, scrapes the resulting pages, and returns the structured content as Haystack Document objects.

You need a Firecrawl API key to use this integration. You can get one at firecrawl.dev.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration allows you to evaluate Haystack pipelines using Flow Judge.

Flow Judge is an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafted for accuracy, speed, and customization.

Read the technical report here.

Installation

For running Flow Judge with vLLM engine:

pip install flow-judge[vllm]
pip install 'flash_attn>=2.6.3' --no-build-isolation

For running Flow Judge with transformers:

pip install flow-judge[hf]

If flash attention:

pip install 'flash_attn>=2.6.3' --no-build-isolation

For running Flow Judge with Llamafile on macOS:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

FunASR is an open-source speech recognition toolkit from Alibaba DAMO Academy that runs entirely locally — no API key required. It supports 50+ languages, speaker diarization, and timestamp extraction.

This integration provides:

FunASRTranscriber: Transcribes audio files to Haystack Document objects. Accepts file paths, Path objects, and ByteStream inputs.

Installation

pip install funasr-haystack

Usage

FunASRTranscriber

FunASRTranscriber transcribes audio files to Haystack Document objects using FunASR models. Models are downloaded from ModelScope on first use and cached in ~/.cache/modelscope.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Future AGI is an open-source e2e agent engineering and optimization platform that helps you ship self-improving AI agents. The traceAI-haystack package auto-instruments Haystack pipelines and exports spans to any OTLP-compatible backend (Future AGI, Jaeger, Datadog, etc.).

Once registered, every component run, generator call, retriever query, and pipeline execution is captured as a structured OpenTelemetry span — no per-component wiring required.

Installation

pip install traceAI-haystack

For the example pipeline below, you also need:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Gandr is a text-to-speech API. This integration adds a GandrTTS component that turns text into WAV audio inside a Haystack pipeline, and optionally writes it to disk.

Six stock voices, output at 8 kHz to 24 kHz, and every render is watermarked.

Installation

pip install gandr-haystack

Usage

Set GANDR_API_KEY in your environment. Keys start with gnd_, and a free key with 100,000 characters is available at gandr.ai without a card.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The GitHub integration for Haystack provides a set of components and tools to interact with GitHub repositories, issues, and pull requests. It enables you to view repository contents, manage issues, create pull requests, and more within your Haystack agents and pipelines.

Some of the components and tools in this integration require GitHub authentication with a personal access token. For example, authentication is required to post a comment on GitHub, fork a repository, or open a pull request. You can create a (fine-graind personal access token)[https://github.com/settings/personal-access-tokens] or a classic personal access token on GitHub and then expose it via an environment variable called GITHUB_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

🚧 This integration uses a deprecated SDK.

We recommend switching to the new Google GenAI integration instead.

Google AI is a machine learning (ML) platform that lets you train and deploy ML models and AI applications, and customize large language models (LLMs) for use in your AI-powered applications. This integration enables the usage of Google generative models via Google AI Studio.

Haystack supports all the available multimodal Gemini models for tasks such as text generation, function calling, visual question answering, code generation, and image captioning.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration brings files from Google Drive into your Haystack pipelines through the Google Drive API v3. It ships two components:

GoogleDriveRetriever — runs a full-text search over the user’s Drive (and optionally shared drives) via the files.list endpoint and returns one Haystack Document per matching file. Each document carries resource metadata (file_name, file_id, web_url, mime_type, file_extension, author, timestamps). By default the content is the file description or name; set include_content=True to export native Google Docs/Sheets/Slides to text and use that as the content. Binary files (PDF, DOCX, …) are never downloaded.
GoogleDriveFetcher — downloads the full content of Drive files and returns them as ByteStreams, ready for a downstream converter. Binary files are downloaded as-is, native Google Docs/Sheets/Slides are exported (by default to DOCX/XLSX/PPTX), and folders or non-downloadable Google types are skipped. Feed it the retriever’s documents or a list of file ids / Drive URLs.

The two components are designed to work together — and with the OAuth integration for authentication — but each can be used on its own.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Google Gen AI provides access to Google’s Gemini models through the new Google Gen AI SDK. This integration enables the usage of Google’s latest generative models via the updated API interface. Google Gen AI is compatible with both the Gemini Developer API and the Vertex AI API.

Haystack supports the latest Gemini models for tasks such as chat completion, function calling, streaming responses and embedding generation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

🚧 This integration uses a deprecated SDK.

We recommend switching to the new Google GenAI integration instead.

Vertex AI is a machine learning (ML) platform that lets you train and deploy ML models and AI applications, and customize large language models (LLMs) for use in your AI-powered applications. This integration enables the usage of generative models and embedding models through Vertex AI API on Google Cloud Platform (GCP).

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Groq is an AI company that has developed Language Processing Unit (LPU), a high-performance engine designed for fast inference of Large Language Models.

To start using Groq, sign up for an API key here. This will give you access to Groq API, which offers rapid inference of open Language Models like Mixtral and Llama 3.

Usage

Groq API is OpenAI compatible, making it easy to use in Haystack via OpenAI Generators.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

You can use HanLP (Han Language Processing) in your Haystack pipelines or as a standalone component for Chinese text processing. HanLP is a comprehensive NLP library for Chinese language processing that provides advanced tokenization, sentence segmentation, and other linguistic analysis capabilities.

The integration provides a specialized ChineseDocumentSplitter component that understands the unique characteristics of Chinese text, such as the lack of spaces between words and the multi-character nature of Chinese words.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Hetzner Inference API serves open-weight LLMs from Hetzner’s European data centers behind an OpenAI-compatible REST API. Once installed, you get access to the HetznerChatGenerator, which lets you call any of the models Hetzner serves.

Two models are available, both with a 262,144-token context window and both accepting images alongside text:

Qwen/Qwen3.6-35B-A3B-FP8
Qwen3.8-27B

In order to follow along with this guide, you’ll need a Hetzner API token for the Inference API. Add it as an environment variable, HETZNER_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Hindsight is an open-source long-term memory engine for AI agents. It stores facts, preferences, and project context across sessions, then retrieves what’s relevant later — and can go a step further by reflecting: synthesizing a query-focused summary from many memories rather than returning raw snippets.

The hindsight-haystack package exposes Hindsight as ready-made Haystack Tool instances, so any Haystack Agent can read and write durable memory. It provides three tools:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

With this integration, you can use models through Hugging Face APIs:

Serverless Inference API (Inference Providers): access many models from different providers through a unified API.
Inference Endpoints: deploy models on dedicated, fully managed infrastructure.
Self-hosted Text Generation Inference (TGI) and Text Embeddings Inference (TEI) servers.

Haystack supports Hugging Face models in other ways too:

Hugging Face Transformers for local models (LLMs, extractive QA, classification, NER)
Sentence Transformers for local embedding and ranking models
Optimum for high-performance inference with ONNX Runtime

Installation

pip install huggingface-api-haystack

Usage

Unless you are using a self-hosted TGI/TEI server, set your Hugging Face token as the HF_API_TOKEN or HF_TOKEN environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Transformers is Hugging Face’s library for state-of-the-art machine learning models. With this integration, you can run models from the Hugging Face Hub locally, on your own machine, in your Haystack pipelines.

Haystack supports Hugging Face models in other ways too:

Sentence Transformers for local embedding and ranking models
Hugging Face API to call models via Inference Providers, Inference Endpoints, or self-hosted TGI/TEI
Optimum for high-performance inference with ONNX Runtime

Installation

pip install transformers-haystack

Usage

Components

This integration provides several components that run Transformers models locally:

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

IBM Db2 Document Store for Haystack

Installation

IBM Db2 (version 12.1.2 and later) provides a native VECTOR data type that adds support for vector similarity search directly inside the database, allowing Db2 to act as a fully featured vector store while keeping documents, embeddings, and metadata within your existing enterprise database.

For more information on Db2 vector capabilities, visit the IBM Db2 product page.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

iFlytek Spark provides chat-completion models through an OpenAI-compatible API. This integration brings iFlytek Spark to Haystack with IFlytekChatGenerator, which builds on Haystack’s OpenAIChatGenerator.

Use this integration when you want to plug Spark models such as generalv3.5, 4.0Ultra, or lite into an existing Haystack pipeline with minimal changes.

Installation

pip install iflytek-haystack

Usage

IFlytekChatGenerator

Get an API password from the iFlytek open platform console and set it as the IFLYTEK_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

An integration of the InterSystems IRIS database with Haystack. In IRIS, the native VECTOR(DOUBLE, N) type is used for storing document embeddings, and the VECTOR_COSINE function enables high-performance dense retrievals using SIMD operations.

The library allows using InterSystems IRIS as a DocumentStore, implementing the required Protocol methods. You can start working with the implementation by importing it from the package:

from intersystems_iris_haystack import IRISDocumentStore

In addition to the IRISDocumentStore, the library includes the following Haystack components which can be used in a pipeline:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Isaacus is a foundational legal AI research company building AI models, apps, and tools for the legal tech ecosystem.

Isaacus offers first-class support for Haystack via the isaacus-haystack package, providing embedders optimized for legal retrieval—most notably Kanon 2, a high-performing legal embedding model (see the Kanon 2 overview and the Massive Legal Embedding Benchmark).

Installation

pip install isaacus-haystack

Usage

Learn more about the embedding models in Isaacus Embeddings API docs

Mon, 01 Jan 0001 00:00:00 +0000

This integration allows users of Haystack to seamlessly use Jina AI’s jina-embeddingsand reranking models in their pipelines. Haystack also integrates the Jina Reader API.

Jina AI is a multimodal AI company, with a vision to revolutionize the way we interpret and interact with information with its prompt and model technologies.

Jina AI offers several models so people can use and chose whatever fits best to their needs:

Model	Dimension	Language	MRL (matryoshka)	Context
jina-embeddings-v3	1024	Multilingual (89 languages)	Yes	8192
jina-embeddings-v2-base-en	768	English	No	8192
jina-embeddings-v2-base-de	768	German & English	No	8192
jina-embeddings-v2-base-es	768	Spanish & English	No	8192
jina-embeddings-v2-base-zh	768	Chinese & English	No	8192

Recommended Model: jina-embeddings-v3 :

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Keenable is a web search and page-fetch API built for AI agents. This integration provides two components:

KeenableWebSearch: searches the web and returns results as Haystack Document objects plus their URLs (links), the same output shape as the SerperDevWebSearch / SearchApiWebSearch integrations, so it drops into existing web-search pipelines.
KeenableFetcher: fetches one or more URLs and returns their main content as Haystack Document objects (extracted to clean markdown server-side, so you don’t need a separate LinkContentFetcher + HTMLToDocument step).

Keyless by default. With no API key, the components use Keenable’s keyless public endpoints — no signup required to try it. Set a KEENABLE_API_KEY to use the authenticated endpoints (required for mode="realtime" and for higher rate limits).

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Kreuzberg is a document intelligence framework with a Rust core that extracts text, tables, and metadata from 91+ file formats — entirely locally with no external API calls.

This integration provides KreuzbergConverter, a Haystack component that converts files into Haystack Document objects with rich metadata. It supports parallel batch extraction using Rust’s rayon thread pool for high throughput.

Supported format categories:

Documents: PDF, DOCX, DOC, PPTX, PPT, XLSX, XLS, ODT, ODS, ODP, RTF, Pages, Keynote, Numbers, and more
Images (via OCR): PNG, JPEG, TIFF, GIF, BMP, WebP, JPEG 2000, SVG
Text/Markup: Markdown, HTML, XML, LaTeX, Typst, JSON, YAML, reStructuredText, Jupyter notebooks
Email: EML, MSG (with attachment extraction)
Archives: ZIP, TAR, GZIP, 7Z (extracts and processes contents recursively)
eBooks & Academic: EPUB, BibTeX, DocBook, JATS

Installation

pip install kreuzberg-haystack

Usage

Components

This integration introduces one component:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

LanceDB-Haystack is an embedded LanceDB backed Document Store for Haystack.

Installation

The current simplest way to get LanceDB-Haystack is to install from GitHub via pip:

pip install lancedb-haystack

Usage

import pyarrow as pa
from lancedb_haystack import LanceDBDocumentStore
from lancedb_haystack import LanceDBEmbeddingRetriever, LanceDBFTSRetriever

# Declare the metadata fields schema, this lets us filter using it.
# See: https://arrow.apache.org/docs/python/api/datatypes.html
metadata_schema = pa.struct([
  ('title', pa.string()),    
  ('publication_date', pa.timestamp('s')),
  ('page_number', pa.int32()),
  ('topics', pa.list_(pa.string()))
])

# Create the DocumentStore
document_store = LanceDBDocumentStore(
  database='my_database', 
  table_name="documents", 
  metadata_schema=metadata_schema, 
  embedding_dims=384
)

# Create an embedding retriever
embedding_retriever = LanceDBEmbeddingRetriever(document_store)

# Create a Full Text Search retriever
fts_retriever = LanceDBFTSRetriever(document_store)

See also examples/pipeline-usage.ipynb for a full worked example, and the API Reference.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The langdetect-haystack integration provides two components for language detection in Haystack pipelines, built on top of the langdetect library:

DocumentLanguageClassifier: classifies the language of each document and stores the detected language in the document’s metadata.
TextLanguageRouter: routes a text string to a different output connection depending on its detected language.

Both components take a list of ISO language codes during initialization. If the detected language is not in that list, the document or text is labeled or routed as "unmatched".

Mon, 01 Jan 0001 00:00:00 +0000

Overview

langfuse-haystack integrates tracing capabilities into Haystack pipelines using Langfuse. This package enhances the visibility of pipeline runs by capturing comprehensive details of the execution traces, including API calls, context data, prompts, and more. Whether you’re monitoring model performance, pinpointing areas for improvement, or creating datasets for fine-tuning and testing from your pipeline executions, langfuse-haystack is the right tool for you.

Features

Easy integration with Haystack pipelines
Capture the full context of the execution
Track model usage and cost
Collect user feedback
Identify low-quality outputs
Build fine-tuning and testing datasets

In order to use this integration, sign up for a Langfuse account. See the Langfuse docs for the most up-to-date information about features and pricing.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Lara is an adaptive translation API by translated that combines the fluency and context handling of LLMs with low hallucination and latency. It adapts to domains at inference time using optional context, instructions, translation memories, and glossaries.

Key features:

Translation styles: Choose between faithful, fluid, or creative styles to control the balance between accuracy and natural flow.
Context-aware translation: Provide surrounding text as context to improve translation quality without translating it.
Instruction-guided translation: Use natural-language instructions to guide translations (e.g. “Be formal”, “Use a professional tone”).
Translation memories: Adapt translations to the style and terminology of existing translation memories.
Glossaries: Enforce consistent terminology (e.g. brand names, product terms) across translations.
Reasoning (Lara Think): Enable multi-step linguistic analysis for higher-quality translations.

For more details, see the Lara SDK documentation and the Lara support documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

LibreOfficeFileConverter is a Haystack component that uses LibreOffice’s command-line utility (soffice) to convert office files between formats. It supports documents, spreadsheets, and presentations, and can output ByteStream objects that plug directly into other Haystack components.

Sources can be file paths (str or Path) or ByteStream objects. Both synchronous (run) and asynchronous (run_async) execution modes are supported.

Installation

First, install LibreOffice on your system:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Linkup is a web search API built for LLM and agent applications, returning grounded results with source URLs.

This integration provides:

LinkupWebSearch: Searches the web using the Linkup API and returns results as Haystack Document objects along with source URLs.

You need a Linkup API key to use this integration. You can get one at linkup.so.

Installation

pip install linkup-haystack

Usage

LinkupWebSearch

LinkupWebSearch queries the Linkup Search API and returns results as Haystack Document objects containing the content snippets and metadata (title, URL). Source URLs are also returned separately.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

LiteLLM provides a single, unified interface to over 100 LLM providers, including OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere, Mistral, and Groq. This integration brings that unified interface to Haystack through the LiteLLMChatGenerator, so you can switch between providers by changing only the model string, without rewriting your pipeline.

Model names use the LiteLLM provider/model-name format, for example openai/gpt-4o, anthropic/claude-sonnet-4-20250514, or bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0. For the full list of supported providers and their model identifiers, see the LiteLLM providers documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The Live Tennis API serves live scores, matches and player data across ATP, WTA, Challenger, ITF and juniors.

This integration provides:

LiveTennisMatchFetcher: fetches live, upcoming, or completed matches (optionally a single match by ID; filterable by tour) and returns them as Haystack Document objects. Each Document’s content is a clean human-readable match summary and its meta carries the structured fields (IDs, players, sets/games/points, server, winner).
LiveTennisPlayerSearch: searches players by name (ranked players first) and returns them as Document objects with the same content/meta split.
LiveTennisH2HFetcher: the head-to-head record between two players — the results archive (1968-2022) plus current matches (2023-now) in one Document. BASIC tier.
LiveTennisArchiveFetcher: the results archive — 1,485,752 matches 1968-2022, player bios and career aggregates. BASIC tier.
LiveTennisRankingsFetcher: a published ranking table (ATP, WTA or the ITF circuits), one Document per row, optionally as of a past week. PRO tier.
LiveTennisMatchStatisticsFetcher: in-play statistics for one match (aces, double faults, serve split, hold/break percentages, break points). ULTRA tier.

You need a Live Tennis API key to use this integration (a free tier is available). The components read it from the LIVETENNISAPI_KEY environment variable via Haystack’s Secret, so serialized pipelines never contain the key. Built on the official livetennisapi Python client.

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Llama.cpp is a library written in C/C++ for efficient inference of Large Language models. It uses the efficient quantized GGUF format, dramatically reducing memory requirements and accelerating inference. This means it is possible to run LLMs efficiently on standard machines (even without GPUs).

Installation

Install the llama-cpp-haystack package:

pip install llama-cpp-haystack

Using a different compute backend

The default installation behaviour is to build llama.cpp for CPU on Linux and Windows and use Metal on MacOS. To use other compute backends:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Llama Stack is an open-source framework consisting of AI building blocks and unified APIs that standardizes building AI Apps across different environments.

The LlamaStackChatGenerator allows you to leverage any LLMs made available by inference providers hosted on a Llama Stack server. It abstracts away the specifics of the underlying provider, enabling you to use the same client-side code across different inference backends. For a list of supported providers and configuration details, refer to the Llama Stack documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

llamafile is a project by Mozilla that aims to make open LLMs accessible to developers and users.

To run LLMs locally, simply download a single-file executable (“llamafile”) that contains both the model and the inference engine and runs locally on most computers.

llamafile can be used on its own to chat with these models, but below we will see how to integrate it with Haystack, to build LLM applications.

Mon, 01 Jan 0001 00:00:00 +0000

Use the LM Format Enforcer to enforce JSON Schema / Regex output of your local models in your haystack pipelines.

Language models are able to generate text, but when requiring a precise output format, they do not always perform as instructed. Various prompt engineering techniques have been introduced to improve the robustness of the generated text, but they are not always sufficient. LM Format Enforcer solves the issues by filtering the tokens that the language model is allowed to generate at every timestep, thus ensuring that the output format is respected, while minimizing the limitations on the language model.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

MariaDB Document Store for Haystack

Installation

MariaDB 11.7+ introduced a native VECTOR datatype with HNSW-based indexing, enabling efficient vector similarity search directly in the database.

To quickly set up a MariaDB instance, you can use Docker:

docker run -d -p 3306:3306 \
  -e MARIADB_ROOT_PASSWORD=secret \
  -e MARIADB_DATABASE=haystack \
  -e MARIADB_USER=haystack \
  -e MARIADB_PASSWORD=secret \
  mariadb:11.7

The mariadb connector is a C extension built from source, so it needs the MariaDB Connector/C system library (mariadb_config):

Mon, 01 Jan 0001 00:00:00 +0000

Overview

MarkItDown is a Python library by Microsoft for converting various file formats into Markdown. It supports a wide range of formats including PDF, Word (.docx), PowerPoint (.pptx), Excel (.xlsx), HTML, images, and more — all processed locally.

This integration provides a MarkItDownConverter component that wraps Microsoft’s MarkItDown library, enabling Haystack users to convert files into Haystack Document objects with Markdown content.

Installation

pip install markitdown-haystack

Usage

Standalone

from haystack_integrations.components.converters.markitdown import MarkItDownConverter

converter = MarkItDownConverter()
result = converter.run(sources=["document.pdf", "report.docx"])
documents = result["documents"]

You can also pass metadata to attach to the resulting documents:

Mon, 01 Jan 0001 00:00:00 +0000

The MastodonFetcher is a simple custom component that fetches the last_k_posts of a given Mastodon username. You can see a demo of this custom component in the 🦄 Should I Follow? space on Hugging Face 🤗.

Overview

This component expects username to be a complete Mastodon username. For example “ tuana@sigmoid.social”. If the provided username is correct and public, MastodonFetcher will return a list of Document objects where the contents are the users latest posts.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

MCP Haystack Integration adds support for the Model Context Protocol (MCP) to Haystack. MCP is an open protocol that standardizes how applications provide context to LLMs, similar to how USB-C provides a standardized way to connect devices.

This integration allows you to easily connect external tools and services to your Haystack pipelines using the Model Context Protocol, enabling more powerful and flexible agentic applications.

Installation

pip install mcp-haystack

Usage

from haystack_integrations.tools.mcp import MCPTool, StreamableHttpServerInfo

# Create an MCP tool that connects to an HTTP server
server_info = StreamableHttpServerInfo(url="http://localhost:8000/mcp")
tool = MCPTool(name="my_tool", server_info=server_info)

# Use the tool
result = tool.invoke(param1="value1", param2="value2")

Examples

Check out the examples directory to see practical demonstrations of how to integrate the MCPTool into Haystack’s tooling architecture. These examples will help you get started quickly with your own agentic applications.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Mem0 provides a memory layer for AI agents and assistants. It helps Haystack applications store user-specific facts, preferences, and project context, then retrieve relevant memories in later conversations.

The mem0-haystack package is part of Haystack Core Integrations and provides:

Mem0MemoryStore: A persistent memory store backed by the Mem0 Cloud API.
Mem0MemoryRetriever and Mem0MemoryWriter: Pipeline components for retrieving and writing ChatMessage memories.
Mem0MemoryRetrieverTool and Mem0MemoryWriterTool: Ready-made tools for memory-augmented Haystack Agents.

More information:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

🚧 This integration is discontinued.

Meta is winding down the hosted Llama API (public preview) on July 6, 2026 — after that date the service shuts down and API requests return a sunset response. Llama models remain available; only the hosted Llama API is being retired. The meta-llama-haystack integration has been archived (see deepset-ai/haystack-core-integrations#3544).

You can keep using Meta Llama models with Haystack via other integrations, for example Ollama, Llama.cpp, Hugging Face API, Amazon Bedrock, or Groq.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration brings content from Microsoft SharePoint and OneDrive into your Haystack pipelines through the Microsoft Graph API. It ships two components:

MSSharePointRetriever — searches SharePoint and OneDrive via the Microsoft Search (Graph) API and returns one Haystack Document per hit. Each document’s content is the search snippet and its meta carries the resource metadata (file_name, web_url, entity_type, timestamps, author info, mime_type, …) plus the SharePoint identifiers a downstream fetcher needs. It does not download the underlying files.
MSSharePointFetcher — downloads the full content of SharePoint and OneDrive items and returns them as ByteStreams, ready for a downstream converter. Files (driveItem) come back as their raw bytes, list items as JSON, and SharePoint pages (sitePage) as HTML. Feed it the retriever’s documents or a list of web_urls.

The two components are designed to work together — and with the OAuth integration for authentication — but each can be used on its own.

Mon, 01 Jan 0001 00:00:00 +0000

Recent Updates

[2025.4.17] Full-text Search with Milvus and Haystack - Learn how to implement full-text and hybrid search in your application using Haystack and Milvus

Installation

pip install --upgrade pymilvus milvus-haystack

Usage

Use the MilvusDocumentStore in a Haystack pipeline as a quick start.

from haystack import Document
from milvus_haystack import MilvusDocumentStore

document_store = MilvusDocumentStore(
    connection_args={"uri": "./milvus.db"},
    drop_old=True,
)
documents = [Document(
    content="A Foo Document",
    meta={"page": "100", "chapter": "intro"},
    embedding=[-10.0] * 128,
)]
document_store.write_documents(documents)
print(document_store.count_documents())  # 1

Different ways to connect to Milvus

For the case of Milvus Lite, the most convenient method, just set the uri as a local file.

document_store = MilvusDocumentStore(
    connection_args={"uri": "./milvus.db"},
    drop_old=True,
)

For the case of Milvus server on docker or kubernetes, it is recommended to use when you are dealing with large scale of data. After starting the Milvus service, you can use the specified uri to connect to the service.

document_store = MilvusDocumentStore(
    connection_args={"uri": "http://localhost:19530"},
    drop_old=True,
)

For the case of Zilliz Cloud, the fully managed cloud service for Milvus, adjust the uri and token, which correspond to the Public Endpoint and Api key in Zilliz Cloud.

from haystack.utils import Secret
document_store = MilvusDocumentStore(
    connection_args={
        "uri": "https://in03-ba4234asae.api.gcp-us-west1.zillizcloud.com",  # Your Public Endpoint
        "token": Secret.from_env_var("ZILLIZ_CLOUD_API_KEY"),  # API key, we recommend using the Secret class to load the token from env variable for security.
        "secure": True
    },
    drop_old=True,
)

Dive deep usage

Prepare an OpenAI API key and set it as an environment variable:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Mirage is a unified virtual filesystem for AI agents: it mounts heterogeneous backends — object storage (S3, GCS, R2), databases (Postgres, MongoDB, Redis), and SaaS apps (Google Drive, Gmail, Slack, GitHub, Notion) — as a single filesystem, so every service speaks the same familiar Unix semantics. An agent can ls, cat, grep and pipe across mounts exactly as it would on local disk, without learning a new API for each backend.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Mistral AI currently provides two types of access to Large Language Models:

An API providing pay-as-you-go access to the latest Mistral models like mistral-embed and mistral-small.
Open-source models available under the Apache 2.0 License, available on Hugging Face which you can use with the HuggingFaceAPIChatGenerator.

For more information on models available via the Mistral API, see the Mistal docs.

In order to follow along with this guide, you’ll need a Mistal API key. Add it as an environment variable, MISTRAL_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

mixedbread ai is an AI start-up that provides open-source, as well as, in-house embedding and reranking models. You can choose from various foundation models to find the one best suited for your use case. More information can be found on the documentation page.

Installation

Install the mixedbread ai integration with a simple pip command:

pip install mixedbread-ai-haystack

Usage

This integration comes with 3 components:

For documents you can use MixedbreadAIDocumentEmbedder and for queries you can use MixedbreadAITextEmbedder. Once you’ve selected the component for your specific use case, initialize the component with the model and the api_key. You can also set the environment variable MXBAI_API_KEY instead of passing the api key as an argument.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

MLflow is an open-source platform for managing the end-to-end machine learning and AI lifecycle. MLflow provides native tracing support for Haystack through its autolog integration, giving you full visibility into your Haystack pipeline execution.

MLflow Tracing offers:

Hierarchical trace visualization of every component, LLM call, retriever step, and pipeline execution
Automatic token usage and cost tracking for each LLM call
Built-in evaluation framework with LLM judges and custom scorers
Prompt versioning and management across your AI applications
Fully open-source with no vendor lock-in, self-host or use Managed MLflow in the cloud

You can learn more about the integration in MLflow’s Haystack integration guide.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

MongoDB is a document database designed to ease application development and scaling. MongoDB Atlas is a multi-cloud database service built by the people behind MongoDB. MongoDB Atlas simplifies deploying and managing your databases while offering the versatility you need to build resilient and performant global applications on the cloud providers of your choice.

You can use MongoDB Atlas’s full-text and semantic search features through MongoDBAtlasFullTextRetriever and MongoDBAtlasEmbeddingRetriever. For a detailed overview of all settings for the MongoDBAtlasDocumentStore, visit the Haystack Documentation.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

MonsterAPI provides access to powerful language models designed for various text generation tasks. With the MonsterAPI integration, you can leverage these models within the Haystack framework for enhanced natural language processing capabilities.

To start using MonsterAPI, sign up for an API key here. This key provides access to the MonsterAPI, which supports rapid inference and customization through various parameters.

Usage

MonsterAPI’s API is OpenAI compatible, making it easy to use within Haystack via OpenAI Generators.

Mon, 01 Jan 0001 00:00:00 +0000

Needle RAG tools for Haystack

This package provides NeedleDocumentStore and NeedleEmbeddingRetriever component for use in Haystack projects.

Usage ⚡️

Get started by installing the package via pip.

pip install needle-haystack-ai

API Keys

We will show you building a common RAG pipeline using Needle tools and OpenAI generator. For using these tools you must set your environment variables, NEEDLE_API_KEY and OPENAI_API_KEY respectively.

You can get your Needle API key from from Developer settings.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

An integration of Neo4j graph database with Haystack v2.0 by deepset. In Neo4j Vector search index is being used for storing document embeddings and dense retrievals.

The library allows using Neo4j as a DocumentStore, and implements the required Protocol methods. You can start working with the implementation by importing it from neo4j_haystack package:

from neo4j_haystack import Neo4jDocumentStore

In addition to the Neo4jDocumentStore the library includes the following haystack components which can be used in a pipeline:

Mon, 01 Jan 0001 00:00:00 +0000

This Haystack component allows you to easily export your Notion pages to Haystack Documents by providing a Notion API token.

Given that the Notion API is subject to some rate limits, this component will automatically retry failed requests and wait for the rate limit to reset before retrying. This is especially useful when exporting a large number of pages. Furthermore, this component uses asyncio to make requests in parallel, which can significantly speed up the export process.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The nvidia-haystack package contains Haystack integrations for chat models, embeddings, and reranking powered by NVIDIA AI Foundation Models and hosted on the NVIDIA API Catalog.

NVIDIA AI Foundation models are community- and NVIDIA-built models that are optimized to deliver the best performance on NVIDIA-accelerated infrastructure. You can use the API to query live endpoints that are available on the NVIDIA API Catalog to get quick results from a DGX-hosted cloud compute environment, or you can download models with NVIDIA NIM, which is included with the NVIDIA AI Enterprise license. The ability to run models on-premises gives your enterprise ownership of your customizations and full control of your IP and AI application.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration provides the OAuthTokenResolver, a component that resolves an OAuth 2.0 access token at pipeline runtime and emits it on its access_token output socket. A downstream component (for example a SharePoint or Google Drive retriever) consumes the token through a normal pipeline connection and never needs to know how it was obtained.

The resolver is a thin wrapper over a pluggable token source that decides where the token comes from. The integration ships three token sources, and you can implement your own:

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

You can use Ollama Models in your Haystack pipelines with the OllamaGenerator.

Ollama is a project focused on running Large Language Models locally. Internally it uses the quantized GGUF format by default. This means it is possible to run LLMs on standard machines (even without GPUs), without having to handle complex installation procedures.

Installation

pip install ollama-haystack

Usage

This integration provides 4 components that allow you to leverage Ollama models:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The haystack-opea integration connects Haystack to OPEA—a collection of containerized microservices for LLMs, embedding, retrieval and reranking. By delegating heavy compute to OPEA services, you can build flexible Retrieval-Augmented Generation (RAG) pipelines that scale across cloud, on-prem and edge deployments.

Key features:

Hardware-agnostic LLM & embedding services.
Easy orchestration of LLM, embedder, retriever, ranker, among others.
Support for local development via Docker Compose or production clusters.

Installation

Install from source:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

You can use OpenAI Models in your Haystack pipelines with the Generators and Embedders. Check out the Whisper Integration for LocalWhisperTranscriber and RemoteWhisperTranscriber.

Installation

pip install haystack-ai

Usage

You can use OpenAI models in various ways:

Embedding Models

You can leverage embedding models from OpenAI through two components: OpenAITextEmbedder and OpenAIDocumentEmbedder.

To create semantic embeddings for documents, use OpenAIDocumentEmbedder in your indexing pipeline. For generating embeddings for queries, use OpenAITextEmbedder. Once you’ve selected the suitable component for your specific use case, initialize the component with the model name and OpenAI API key.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

OpenAPI is a widely used standard for describing REST APIs. The openapi-haystack integration lets your Haystack pipelines and LLMs call any OpenAPI-compliant service: you can invoke endpoints directly from a specification, or turn a spec into tool/function definitions that an LLM can call.

Consider using MCP instead

If you are building new pipelines or agents that need to call external tools and services, prefer the MCPTool. It provides a standardized, well-supported way to expose tools to LLMs and integrates natively with Haystack’s tooling and agents. The OpenAPI components documented below remain available for cases where you must consume an existing OpenAPI specification directly.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

OpenLIT is an OpenTelemetry-native GenAI and LLM application observability tool. It simplifies the integration of observability into GenAI and LLM using Haystack with just a single line of code.

OpenLIT allows you to monitor detailed information about LLM operations, such as API calls, content, prompts, costs, and more. This enables you to gain insights into your models’ performance, identify improvement areas, and reduce costs.

Check out the official documentation for more information.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

OpenRouterChatGenerator lets you call any LLMs available on OpenRouter, including:

OpenAI variants such as openai/gpt-4o
Anthropic’s claude-3.5-sonnet
Community-hosted open-source models (Llama 2, Mixtral, etc.)

For more information on models available via the OpenRouter API, see the OpenRouter docs.

In addition to basic chat completion, the component exposes OpenRouter-specific features:

Provider / model routing – choose fallback models or provider ordering with the generation_kwargs parameter.
Extra HTTP headers – add attribution or tracing headers via extra_headers.

In order to follow along with this guide, you’ll need a OpenRouter API key. Add it as an environment variable, OPENROUTER_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Installation

Use pip to install OpenSearch:

pip install opensearch-haystack

Usage

Once installed, initialize your OpenSearch database to use it with Haystack:

from haystack_integrations.document_stores.opensearch import OpenSearchDocumentStore

document_store = OpenSearchDocumentStore()

Writing Documents to OpenSearchDocumentStore

To write documents to OpenSearchDocumentStore, create an indexing pipeline.

from haystack.components.file_converters import TextFileToDocument
from haystack.components.writers import DocumentWriter

indexing = Pipeline()
indexing.add_component("converter", TextFileToDocument())
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("converter", "writer")
indexing.run({"converter": {"paths": file_paths}})

Hybrid Retriever

This integration also provides a hybrid retriever. The OpenSearchHybridRetriever combines the capabilities of a vector search and a keyword search. It uses the OpenSearch document store to retrieve documents based on both semantic and keyword-based queries.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Installation

pip install osm-integration-haystack

Overview

This repository implements a Haystack component that integrates with OpenStreetMap data through the Overpass API. It allows you to fetch geographic information and convert it into Haystack Documents for use in RAG (Retrieval-Augmented Generation) pipelines.

When you give OSMFetcher a location and radius, it returns a list of nearby points of interest (POIs) as Haystack Documents. It uses the Overpass API to query OpenStreetMap data and converts the results into structured documents with geographic metadata.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration lets you use OpenTelemetry to trace and monitor your Haystack pipelines and agents. It builds on the OpenTelemetry SDK and provides an OpenTelemetryConnector component that, once added to your pipeline, sends Haystack traces to any OpenTelemetry-compatible backend.

Installation

pip install opentelemetry-haystack

Usage

Configure an OpenTelemetry TracerProvider with an exporter, then add the OpenTelemetryConnector to your pipeline without connecting it to any other component. It enables OpenTelemetry tracing for all pipeline operations.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Open WebUI is an open-source chat UI for LLM apps. By exposing your Haystack pipelines and agents through Hayhooks as OpenAI-compatible endpoints, you can use Open WebUI as the frontend: run Hayhooks and Open WebUI (separately or via Docker Compose), then connect Open WebUI to Hayhooks in Settings. You get streaming, optional status and notification events, and optional OpenAPI tool server integration.

For full details, see the Hayhooks Open WebUI integration guide.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Opik is an open source tool that helps you to trace, evaluate and monitor your LLM applications. With the Opik platform, you can:

Debug your pipelines
Automatically evaluate your pipelines with built-in metrics like hallucinations or context relevance
Track the latency and cost of your pipeline runs
Monitor your pipelines in production

You can learn more about the Haystack and Opik integration in Opik’s Haystack integration guide.

Installation

To use the Opik integration with Haystack, install the opik package:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Hugging Face Optimum is an extension of the Transformers library that provides a set of performance optimization tools to train and run models on targeted hardware with maximum efficiency. Using Optimum, you can leverage the ONNX Runtime to automatically export models from the Hugging Face Model Hub and deploy them in pipelines to achieve significant improvements in performance.

Installation

pip install optimum-haystack

Usage

Components

This integration introduces two components: OptimumTextEmbedder and OptimumDocumentEmbedder.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Oracle AI Vector Search is a feature of Oracle Database 23ai and 26ai that provides native vector storage and similarity search using the VECTOR data type — no extensions or plugins required.

This integration provides an OracleDocumentStore and an OracleEmbeddingRetriever that you can use in Haystack pipelines. It supports HNSW indexing for fast approximate search, metadata filtering with the full Haystack filter grammar, wallet-based TLS connections to Oracle Autonomous Database, and async variants for all public methods.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

OrcaRouterChatGenerator lets you call chat models through OrcaRouter, an OpenAI-compatible API gateway.

OrcaRouter routes requests to provider-prefixed models from upstream providers such as OpenAI, Anthropic, Google Gemini, DeepSeek, xAI Grok, Alibaba Qwen, Moonshot Kimi, and MiniMax. Use the live OrcaRouter model catalog or the /v1/models endpoint to see which models your account can access.

This integration provides:

OpenAI-compatible chat generation through the OrcaRouter API at https://api.orcarouter.ai/v1.
Provider-prefixed model IDs such as openai/gpt-4o-mini, google/gemini-2.5-flash, and deepseek/deepseek-chat.
Automatic routing with orcarouter/auto, which lets OrcaRouter choose a live model for the request.
Fallback chains and routing preferences by forwarding OrcaRouter-specific options through generation_kwargs.
Streaming, tool calling, and structured outputs inherited from Haystack’s OpenAIChatGenerator.

To follow along with the examples below, create an OrcaRouter API key and set it as the ORCAROUTER_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

haystack-oxidize-pdf is a Haystack converter backed by oxidize-pdf, a Rust-powered PDF engine with first-class RAG primitives. The parser runs natively (no system dependencies — it ships as a wheel for Linux, macOS and Windows) and exposes element-disjoint semantic chunking, so PDFs become retrieval-ready Document objects without any post-processing.

The chunking contract is enforced by regression tests: no chunk’s text is a substring of another’s, and every source element appears in exactly one chunk. This guarantees no duplicated context leaks into a vector store during ingestion.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

PaddleOCR converts documents and images into structured, AI-friendly data (like JSON and Markdown) with industry-leading accuracy—powering AI applications for everyone from indie developers and startups to large enterprises worldwide.

This integration allows you to use PaddleOCR’s text-recognition and document-parsing capabilities with Haystack.

Components

PaddleOCRVLDocumentConverter. This component extracts text from documents using PaddleOCR’s large model document parsing API.

Initialization

Every component of the PaddleOCR integration requires an access token from PaddlePaddle AI Studio. By default, authentication uses the AISTUDIO_ACCESS_TOKEN environment variable. You can also provide an access_token when initializing each component. The AI Studio access token can be obtained from this page.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The perplexity-haystack package lets you use Perplexity’s Agent API, Embeddings API, and grounded Search API in Haystack pipelines through four components:

PerplexityChatGenerator — chat completions through the Perplexity Agent API (OpenAI-compatible Responses API). Defaults to openai/gpt-5.4; other supported models include openai/gpt-5.5, openai/gpt-4o, anthropic/claude-sonnet-4-6, xai/grok-4-1, and google/gemini-3-flash-preview.
PerplexityTextEmbedder and PerplexityDocumentEmbedder — embeddings through the Perplexity Embeddings API. Defaults to pplx-embed-v1-0.6b; pplx-embed-v1-4b is also available.
PerplexityWebSearch — grounded web search results through the Perplexity Search API.

For more information about the Perplexity API, see the Perplexity docs.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

Perseus Vault is a local-first, single-binary memory engine for AI agents. It stores data in an encrypted (AES-256-GCM) SQLite database with FTS5 full-text and vector search, runs fully offline, and requires no external vector database, no cloud service, and no API keys. Your data never leaves the machine — which makes it a fit for privacy-sensitive, air-gapped, and cost-constrained deployments.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Pgvector Document Store for Haystack
- Installation
- Usage
- Examples
- License

Installation

pgvector is an extension for PostgreSQL that adds support for vector similarity search.

To quickly set up a PostgreSQL database with pgvector, you can use Docker:

docker run -d -p 5432:5432 -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=postgres pgvector/pgvector:pg17

For more information on how to install pgvector, visit the pgvector GitHub repository.

Use pip to install pgvector-haystack:

pip install pgvector-haystack

Usage

Define the connection string to your PostgreSQL database in the PG_CONN_STR environment variable. For example:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Pinecone is a fast and scalable vector database that you can use in Haystack pipelines with the PineconeDocumentStore.

For a detailed overview of all the available methods and settings for the PineconeDocumentStore, visit the Haystack API Reference.

Installation

pip install pinecone-haystack

Usage

To use Pinecone as your data storage for your Haystack LLM pipelines, you must have an account with Pinecone and an API Key. Once you have those, you can initialize a PineconeDocumentStore for Haystack:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Pixeltable is open-source Python data infrastructure for multimodal AI. It provides persistent, versioned tables that store text, images, video, audio, and documents alongside embeddings and metadata, with incremental computation via computed columns.

This integration provides two components:

PixeltableDocumentStore — a Haystack DocumentStore backed by a Pixeltable table with a built-in embedding index.
PixeltableRetriever — a Haystack Retriever component that performs vector similarity search.

The .table property exposes the underlying Pixeltable table when you need computed columns, version history, or multimodal operations beyond the Haystack Document Store interface.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration provides a Haystack component for PraisonAI, enabling you to run multi-agent AI workflows within your Haystack pipelines. PraisonAI orchestrates multiple AI agents to collaboratively solve complex tasks.

Installation

pip install haystack-praisonai

Usage

Components

This integration introduces the PraisonAIComponent:

PraisonAIComponent: Sends queries to a PraisonAI server and returns agent responses.

Basic Usage

from haystack import Pipeline
from haystack_praisonai import PraisonAIComponent

# Create the component
praisonai = PraisonAIComponent(api_url="http://localhost:8080")

# Use in a pipeline
pipeline = Pipeline()
pipeline.add_component("praisonai", praisonai)

# Run
result = pipeline.run({"praisonai": {"query": "Research the latest AI trends"}})
print(result["praisonai"]["response"])

Using a Specific Agent

from haystack_praisonai import PraisonAIComponent

component = PraisonAIComponent(api_url="http://localhost:8080")
result = component.run(query="Write an article about AI", agent="writer")
print(result["response"])

Configuration

Parameter	Type	Default	Description
`api_url`	str	`http://localhost:8080`	PraisonAI server URL
`timeout`	int	`300`	Request timeout in seconds

Prerequisites

Start a PraisonAI server:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Microsoft Presidio is an open-source library for PII detection and anonymization using NLP-based entity recognition.

presidio-haystack provides three Haystack components:

Component	Input	Purpose
`PresidioDocumentCleaner`	`list[Document]`	Replace PII in document text with entity type placeholders
`PresidioTextCleaner`	`list[str]`	Replace PII in plain strings — useful for sanitizing user queries
`PresidioEntityExtractor`	`list[Document]`	Detect PII and store entities as structured document metadata

All components run locally — no external API required. Presidio uses spaCy NLP models under the hood.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

Prior Labs is the team behind TabPFN, a foundation model for tabular data. They expose TabPFN as a cloud service via the Model Context Protocol (MCP), enabling Haystack agents to run tabular machine learning without writing any ML code.

The MCP server exposes five tools (as of March 2026):

Tool	Description
`upload_dataset`	Upload a CSV file (train or test) and receive a `dataset_id` for use in subsequent calls
`fit_and_predict_from_dataset`	Train a TabPFN model on an uploaded training file and predict on an uploaded test file
`predict_from_dataset`	Run prediction with a previously trained model on an uploaded test file
`fit_and_predict_inline`	Train and predict on small arrays already present in the conversation
`predict_inline`	Predict with a previously trained model on inline arrays

Both classification and regression tasks are supported. For small datasets shared directly in the conversation the agent uses the inline tools, while for larger file-based datasets it calls upload_dataset first.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Pyversity is a library for result diversification in retrieval pipelines. This integration wraps pyversity’s diversification algorithms as a Haystack component, making it easy to balance relevance and diversity in your search results.

The PyversityRanker reranks documents by trading off between relevance scores and embedding-based diversity using strategies such as Maximal Marginal Relevance (MMR) or Determinantal Point Processes (DPP).

Installation

pip install pyversity-haystack

Usage

Components

This integration introduces one component:

Mon, 01 Jan 0001 00:00:00 +0000

An integration of Qdrant vector database with Haystack by deepset.

The library finally allows using Qdrant as a document store, and provides an in-place replacement for any other vector embeddings store. Thus, you should expect any kind of application to be working smoothly just by changing the provider to QdrantDocumentStore.

🎥 Learn how to build a movie recommendation assistant powered by Haystack and Qdrant in 👩🏻‍🍳 Notebook: Agent-Powered Retrieval with Haystack.

Installation

qdrant-haystack might be installed as any other Python library, using pip or poetry:

Mon, 01 Jan 0001 00:00:00 +0000

Overview
Installation
Usage
- 3.1 Evaluation with Integrated RagasEvaluator Component
  - 3.1.1 Importing Required Libraries and Setting Up Environment
  - 3.1.2 Getting the Dataset
  - 3.1.3 Initializing RAG Pipeline Components
  - 3.1.4 Configuring RagasEvaluator Component
  - 3.1.5 Building and Connecting the RAG Pipeline
  - 3.1.6 Running the Pipeline
- 3.2 Standalone Evaluation of the RAG Pipeline
  - 3.2.1 Setting Up a Basic RAG Pipeline
  - 3.2.2 Extracting Outputs for Evaluation
  - 3.2.3 Evaluating the Pipeline Using Ragas EvaluationDataset

Overview

Ragas is an open source framework for model-based evaluation to evaluate your LLM applications by quantifying their performance on aspects such as correctness, tonality, hallucination, fluency, etc. More information can be found on the documentation page.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

ray-haystack is a python package which allows running Haystack pipelines on Ray in a distributed manner. The package provides the same API to build and run Haystack pipelines, but under the hood, components are being distributed to remote nodes for execution using Ray primitives. Specifically, Ray Actor is created for each component in a pipeline to run its logic.

The purpose of this library is to showcase the ability to run Haystack in a distributed setup with Ray featuring its options to configure the payload, e.g:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Respan is an observability and AI gateway platform for LLM applications. The respan-instrumentation-haystack package instruments Haystack pipelines through OpenTelemetry and exports pipeline, component, and LLM spans to Respan.

With Respan and Haystack, you can:

Trace Haystack pipeline runs, component calls, prompt content, responses, token usage, latency, and cost
Route Haystack’s OpenAI-compatible generators through the Respan gateway with only a Respan API key
Use Respan-managed prompts from Haystack by passing a managed prompt_id and variables through generation_kwargs.extra_body.prompt

Installation

pip install respan-ai respan-instrumentation-haystack haystack-ai

Create a Respan API key in the Respan platform, then set:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Rubric Protocol produces independently verifiable evidence of what your AI pipelines did and when. Every attestation is signed with ML-DSA-65 (FIPS 204) post-quantum cryptography and anchored to Hedera Consensus Service — a public, neutral ledger — so anyone can verify a pipeline’s output years later at rubric-protocol.com/verify, without trusting your logs or your vendor.

This matters wherever two parties need to agree on what an AI system did: regulatory examinations (EU AI Act Annex IV, SR 26-2, Illinois SB 315), insurance claims, vendor disputes, and internal audit. Unlike mutable application logs, an anchored attestation cannot be silently edited after the fact.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

SambaNova is an AI company that develops SN40L Reconfigurable Dataflow Unit (RDU), a processor that provides native dataflow processing and high-performance for fast inference of Large Language Models.

To start using SambaNova, sign up for an API key here. This will give you access to SambaNova Cloud API, which offers rapid inference of open Language Models like Llama 3 and Qwen.

Usage

SambaNova Cloud API is OpenAI compatible, making it easy to use in Haystack via OpenAI Generators.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Scavio is a unified search API built for AI agents. One key reaches real-time data across Google, Amazon, Walmart, Target, eBay, Home Depot, YouTube, Reddit, TikTok, TikTok Shop, Instagram, X, Threads, Kuaishou, LinkedIn, Indeed, Glassdoor, Zillow, Redfin, Booking.com, Airbnb, Tripadvisor, Yelp, the Apple App Store, Google Play, SEC EDGAR, Companies House, G2, Capterra, Google Ads Transparency and the Meta Ad Library, plus an extract endpoint that turns any URL into HTML, Markdown or plain text. Everything comes back as structured JSON, with no scraping or proxies to run.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

ScrapeUnblocker renders web pages in a real browser behind anti-bot protections such as Cloudflare, DataDome, PerimeterX and Akamai. Use it when an ordinary HTTP request returns a block page, a captcha, or an empty JavaScript shell instead of the content you need.

The integration provides two components:

ScrapeUnblockerFetcher - fetch URLs and return one Document per page, as HTML or AI-parsed structured JSON
ScrapeUnblockerWebSearch - search Google and return the organic results as Documents

You need a ScrapeUnblocker API key to use both. Get one at scrapeunblocker.com and expose it as SCRAPEUNBLOCKER_API_KEY, which both components read by default.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The SearchApiWebSearch component allows you to perform web searches using the SearchApi service. It retrieves relevant snippets and URLs that can be used directly in your Haystack applications, such as Retrieval-Augmented Generation (RAG) pipelines or with Haystack Agents.

This component is part of the searchapi-haystack integration package, maintained in haystack-core-integrations.

When you pass a query to SearchApiWebSearch, it returns a list of URLs and text snippets that are most relevant to your search.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Sentence Transformers is a library for state-of-the-art embedding and reranking models. With this integration, you can run Sentence Transformers compatible models from the Hugging Face Hub locally, on your own machine, in your Haystack pipelines.

Haystack supports Hugging Face models in other ways too:

Hugging Face Transformers for other local models (LLMs, extractive QA, classification, NER)
Hugging Face API to call models via Inference Providers, Inference Endpoints, or self-hosted TGI/TEI
Optimum for high-performance inference with ONNX Runtime

Installation

pip install sentence-transformers-haystack

Usage

Components

Haystack provides several components based on Sentence Transformers:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The SerperDevWebSearch component allows you to perform web searches using the Serper.dev API. It retrieves relevant snippets and URLs that can be used directly in your Haystack applications, such as Retrieval-Augmented Generation (RAG) pipelines or with Haystack Agents.

This component is part of the serperdev-haystack integration package, maintained in haystack-core-integrations.

When you pass a query to SerperDevWebSearch, it returns a list of URLs and text snippets that are most relevant to your search.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Serpex is a unified web search API that provides access to multiple search engines through a single interface. This Haystack integration allows you to seamlessly incorporate web search results into your Haystack RAG (Retrieval-Augmented Generation) pipelines and AI applications.

Website: serpex.dev

Key Features

🔍 Multi-Engine Support: Switch between Google, Bing, DuckDuckGo, Brave, Yahoo, and Yandex
⚡ High Performance: Fast and reliable API with automatic retries
🎯 Rich Results: Get organic search results with titles, snippets, and URLs
🕒 Time Filters: Filter results by day, week, month, or year
🔒 Type-Safe: Fully typed with comprehensive type hints
📝 Haystack Native: Seamless integration with Haystack 2.0+ components

Installation

pip install serpex-haystack

Usage

Basic Usage

from haystack.utils import Secret
from haystack_integrations.components.websearch.serpex import SerpexWebSearch

# Initialize the component
web_search = SerpexWebSearch(
    api_key=Secret.from_env_var("SERPEX_API_KEY"),
    engine="google",  # Options: google, bing, duckduckgo, brave, yahoo, yandex
)

# Perform a search
results = web_search.run(query="What is Haystack AI?")

# Access the results
for doc in results["documents"]:
    print(f"Title: {doc.meta['title']}")
    print(f"URL: {doc.meta['url']}")
    print(f"Snippet: {doc.content}\n")

Agent Example

Use Serpex in a Haystack agent for dynamic web search:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

An integration of the SingleStore database with Haystack by deepset. In SingleStore, a vector index is used to store document embeddings and support efficient approximate nearest neighbor (ANN)–based dense retrieval for semantic use cases such as RAG and similarity search. In contrast, a full-text search index ( VERSION 2) is used to perform Lucene-compatible, BM25-scored keyword and phrase searches over text and JSON content for traditional text-based retrieval.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Snowflake table retriever for Haystack
- Installation
- Usage
- Examples
- License

Installation

Use pip to install Snowflake:

pip install snowflake-haystack

Usage

Once installed, initialize the SnowflakeTableRetriever to use it with Haystack. The integration supports multiple authentication methods including Multi-Factor Authentication (MFA).

Authentication Methods

Password Authentication

from haystack_integrations.components.retrievers.snowflake import SnowflakeTableRetriever
from haystack.utils import Secret

# Traditional password authentication
executor = SnowflakeTableRetriever(
    user="<ACCOUNT-USER>",
    account="<ACCOUNT-IDENTIFIER>",
    authenticator="SNOWFLAKE",
    api_key=Secret.from_env_var("SNOWFLAKE_API_KEY"),
    warehouse="<WAREHOUSE-NAME>",
)

Key-pair Authentication (MFA)

# JWT-based authentication using private key files
executor = SnowflakeTableRetriever(
    user="<ACCOUNT-USER>",
    account="<ACCOUNT-IDENTIFIER>",
    authenticator="SNOWFLAKE_JWT",
    private_key_file=Secret.from_env_var("SNOWFLAKE_PRIVATE_KEY_FILE"),
    private_key_file_pwd=Secret.from_env_var("SNOWFLAKE_PRIVATE_KEY_PWD"),  # Optional if key is encrypted
    warehouse="<WAREHOUSE-NAME>",
)

OAuth Authentication (MFA)

# OAuth-based authentication
executor = SnowflakeTableRetriever(
    user="<ACCOUNT-USER>",
    account="<ACCOUNT-IDENTIFIER>",
    authenticator="OAUTH",
    oauth_client_id=Secret.from_env_var("SNOWFLAKE_OAUTH_CLIENT_ID"),
    oauth_client_secret=Secret.from_env_var("SNOWFLAKE_OAUTH_CLIENT_SECRET"),
    oauth_token_request_url="<TOKEN-REQUEST-URL>",
    warehouse="<WAREHOUSE-NAME>",
)

Authentication Parameters

The SnowflakeTableRetriever supports three authentication methods:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Soofi ( Sovereign Open Source Foundation Models) is a German research consortium building open European foundation models for industrial AI. The first model family, Soofi S, is a ~30B-parameter hybrid Mamba-2 / Mixture-of-Experts model (about 3.5B active parameters) with a strong focus on English and German.

Preview checkpoints are on the Soofi-Project Hugging Face organization. Three models are available:

Model	Role
`Soofi-S-Instruct-Preview`	Instruct model
`Soofi-S-Isar-Preview`	Reasoning model
`Soofi-S-Rhine-Preview`	Reasoning model

Each model ships in several quantization variants:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

spaCy is a popular open-source library for Natural Language Processing in Python. The spacy-haystack integration provides the SpacyNamedEntityExtractor, which uses spaCy models to recognize named entities — such as people, organizations, and locations — and attach them to your documents.

Installation

Install the spacy-haystack package:

pip install spacy-haystack

Usage

Components

This integration provides one component:

SpacyNamedEntityExtractor: annotates named entities in documents using a spaCy model.

When initializing it, you must set a model. Optionally, you can pass pipeline_kwargs (forwarded to the spaCy pipeline) and a device to run the model on.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

The SQLAlchemy integration provides a SQLAlchemyTableRetriever component that connects to any SQLAlchemy-supported database, executes a SQL query, and returns results as a Pandas DataFrame and an optional Markdown-formatted table string.

Supported backends include PostgreSQL, MySQL, MariaDB, SQLite, MSSQL, and Oracle — anything SQLAlchemy supports works out of the box.

This component is designed for Text-to-SQL pipelines where an LLM generates a SQL query and the retriever fetches the corresponding rows for downstream processing.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

STACKIT provides access to Large Language Models via an API. This Haystack integration introduces a STACKITChatGenerator component to use that API and the chat completion models served by STACKIT, such as neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8, neuralmagic/Mistral-Nemo-Instruct-2407-FP8, neuralmagic/Meta-Llama-3.1-8B-Instruct-FP8. In addition, there are a STACKITTextEmbedder and a STACKITDocumentEmbedder component for embedding tasks with intfloat/e5-mistral-7b-instruct. In order to follow along with this guide, you’ll need a STACKIT API key. Add it as an environment variable, STACKIT_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Supabase is an open-source Postgres platform. The supabase-haystack package provides three sets of components for building Haystack pipelines:

pgvector — dense embedding and keyword retrieval via the pgvector extension (pre-installed on Supabase).
PGroonga — full-text BM25 search via the pgroonga extension (no embeddings required).
Supabase Storage — download files from a Supabase Storage bucket into ByteStream objects ready for indexing.

The pgvector components are a thin wrapper around pgvector-haystack, inheriting all of its functionality: three vector similarity functions (cosine_similarity, inner_product, l2_distance), exact or HNSW search, metadata filtering, and keyword retrieval via PostgreSQL’s ts_rank_cd. The two Supabase-specific defaults are that the connection string is read from SUPABASE_DB_URL and that create_extension is False (Supabase enables pgvector for you).

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Superlinked’s Search Inference Engine (SIE) is a self-hosted inference server for embeddings, reranking, and extraction. The sie-haystack package provides Haystack 2.0 components that route requests through a single SIE endpoint for 85+ embedding models (dense, sparse, multivector/ColBERT, multimodal), cross-encoder reranking, and zero-shot entity, relation, classification, and object-detection extraction.

All components live under the standard Haystack integrations namespace: haystack_integrations.components.{embedders,rankers,extractors}.sie.

Start a local SIE server with Docker before running any of the examples below:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Synap is a managed long-term memory layer for AI agents. It runs a full extraction pipeline on every conversation turn — automatically identifying facts, preferences, episodes, emotions, and temporal events — and retrieves only what is semantically relevant to the current query.

The maximem-synap-haystack package provides a Haystack-native memory store that follows the same shape as mem0-haystack:

SynapMemoryStore: A persistent memory store backed by the Synap API. Owns all SDK interaction (add_memories / search_memories / search_memories_as_single_message).
SynapMemoryRetriever and SynapMemoryWriter: Pipeline @component classes for retrieving memories as ChatMessage objects and writing conversation turns to the store.
SynapRetriever: An additional @component that returns memories as Document objects for classic RAG-style pipelines.

Memory is scoped to the user_id and customer_id you provide, ensuring strict isolation in multi-tenant applications.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Tavily is an AI-powered search API built for LLM applications. It returns high-quality, structured results with relevant content and source URLs — without the noise of traditional search engines.

This integration provides:

TavilyWebSearch: Searches the web using the Tavily API and returns results as Haystack Document objects along with source URLs.

You need a Tavily API key to use this integration. You can get one at tavily.com.

Installation

pip install tavily-haystack

Usage

TavilyWebSearch

TavilyWebSearch queries the Tavily Search API and returns results as Haystack Document objects containing the content snippets and metadata (title, URL). Source URLs are also returned separately.

Mon, 01 Jan 0001 00:00:00 +0000

Overview
Installation
Usage
Features
Support
License

Overview

Add deterministic governance to any Haystack pipeline or agent. No LLM in the governance path — all policy evaluation is deterministic, adding <2ms latency.

v0.2.0 adds native support for Haystack 3.0 Agent Hooks (before_tool).

Installation

pip install tealtiger-haystack

Usage

Agent Hooks (Haystack 3.0+) — Recommended for Agents

Use TealTigerGovernanceHook as a before_tool hook to enforce governance on every tool call an agent makes:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Thunderbolt is an open-source, cross-platform AI client developed by MZLA Technologies (Thunderbird). It runs on web, iOS, Android, Mac, Linux, and Windows, and works with any OpenAI-compatible model endpoint — including self-hosted ones.

By exposing your Haystack pipeline through Hayhooks as an OpenAI-compatible endpoint, you can connect Thunderbolt to your pipeline and interact with it from any device — without building a frontend yourself.

Thunderbolt is designed for enterprise on-prem deployments but can be self-hosted locally for development and testing.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The tika-haystack integration provides TikaDocumentConverter, a component that converts files of different types (PDF, DOCX, HTML, RTF, and many others) into Haystack Document objects using Apache Tika.

Apache Tika is a content analysis toolkit that detects and extracts metadata and text from many file formats. The component requires a running Tika server to parse documents.

This component was previously part of Haystack core and now lives in the tika-haystack integration package, maintained in haystack-core-integrations.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

You can use the Takeoff inference server to deploy local models efficiently in your Haystack pipelines. Takeoff is a state-of-the art inference server focused on deploying openly available language models at scale. It can run LLMs on local machines with consumer GPUs, and on cloud infrastructure.

The TakeoffGenerator component in Haystack is a wrapper around the Takeoff server API, and can be used to serve takeoff-deployed models efficiently in Haystack pipelines.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Once installed you will have access to TogetherAIGenerator and TogetherAIChatGenerator that allow you to call any LLMs available on TogetherAI, including:

OpenAI variants such as oopenai/gpt-oss-120B
deepseek-ai’s deepseek-ai/DeepSeek-R1
Other open-source models (Llama 2, Mixtral, etc.)

For more information on models available via the TogetherAI API, see the TogetherAI docs.

In order to follow along with this guide, you’ll need a TogetherAI API key. Add it as an environment variable, TOGETHER_API_KEY.

Mon, 01 Jan 0001 00:00:00 +0000

Table of Contents

Overview

Tonic Textual is a PII detection and transformation platform powered by transformer-based NER models that identify 46+ entity types across 50+ languages.

textual-haystack provides two Haystack components:

Component	Purpose
`TonicTextualDocumentCleaner`	Synthesize or tokenize PII in document content before ingestion
`TonicTextualEntityExtractor`	Extract PII entities and store them as structured document metadata

Use the document cleaner to sanitize documents before they enter your RAG pipeline — replacing real PII with realistic synthetic data or reversible placeholder tokens. Use the entity extractor to detect PII and attach structured metadata (entity type, value, location, confidence) to documents for hybrid retrieval, auditing, or compliance workflows.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

TopK is a hosted database powering fast vector search, keyword search (BM25), hybrid search and multi-vector search.

This integration ships with TopK Document Store and five retrievers you can use to best fit your use case:

TopKSemanticRetriever — semantic search with server-side embedding, no embedder component needed
TopKBM25Retriever — keyword search using BM25 scoring
TopKEmbeddingRetriever — dense vector search with your own embedding model
TopKHybridRetriever — combines vector and BM25 scores in a single query
TopKMetadataRetriever — filter documents by metadata fields

Installation

pip install topk-haystack

Prerequisites

Before you set up TopK Document Store in Haystack, you’ll need:

Mon, 01 Jan 0001 00:00:00 +0000

OpenLLMetry

OpenLLMetry

OpenLLMetry is an open-source Python package built and maintained by Traceloop that instruments your Haystack-based applications with OpenTelemetry. This gives you full visibility to your LLM app, right in your existing observability stack. You can also connect this to Traceloop to get quality evaluation metrics and LLM-specific capabilities like Prompt Playground.

Installation

pip install traceloop-sdk

Example

Basic integration is as simple as adding one line to your code:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Trafilatura is a cutting-edge Python package and command-line tool designed to gather text on the Web and simplify the process of turning raw HTML into structured, meaningful data. Its extraction component is seamlessly integrated into Haystack.

Going from HTML bulk to essential parts can alleviate many problems related to text quality by focusing on the actual content and avoiding the noise, which is beneficial for LLM applications.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Unstructured Transform turns any file into agent-ready data, called directly from your agent with no separate pipeline to wire up. It is Unstructured’s document-processing pipeline, exposed as a hosted Model Context Protocol server at https://mcp.transform.unstructured.io. Drop in a PDF, spreadsheet, scan, or email and get back partitioned, enriched, chunked, and embedded output ready for RAG, vector stores, or agent memory, with tables and layout intact.

The pipeline itself runs asynchronously as a job: submit a file for processing, poll until it’s done, then fetch the rendered result; a separate helper mints an upload URL for files that aren’t already reachable over HTTPS. Unstructured adds tools and capabilities to this server as they ship new features, so rather than list exact tool names and a fixed count here (which would go stale the next time they do), the snippets below discover the live toolset at connect time and let the agent match tools to the task by their description.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

TwelveLabs builds video-native foundation models. This integration brings two of them to Haystack:

Marengo — a multimodal embedding model that maps text, images, audio, and video into a single shared vector space. Embeddings produced from text are directly comparable (cosine similarity) with embeddings of images, audio, and video, which enables cross-modal retrieval (for example, searching a video collection with a text query).
Pegasus — a video-language model that analyzes a video on the fly (its visuals and its own audio via ASR) and returns text, so a video becomes a Document whose content is the analysis — no frame extraction or separate transcription step.

Get a free API key at playground.twelvelabs.io and set it as the TWELVELABS_API_KEY environment variable.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Component for the Haystack LLM framework to convert files and directories into Documents using the Unstructured API.

Unstructured provides ETL tools for LLMs, extracting text and other information from various file formats. See supported file types for more details.

Installation

To install the Unstructured File Converter, run:

pip install unstructured-fileconverter-haystack

Usage

Connecting to the Unstructured API

Hosted API

The Unstructured API is available in both free and paid versions: Unstructured Serverless API or Free Unstructured API.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Upstash Vector is a serverless, pay-as-you-go vector database that you can use in Haystack pipelines with the UpstashDocumentStore. It requires zero infrastructure — no Docker containers, no servers, no clusters to manage.

This integration provides three components:

Component	Description
`UpstashDocumentStore`	Full-featured document store backed by Upstash Vector
`UpstashEmbeddingRetriever`	Dense retrieval using cosine/dot-product similarity
`UpstashHybridRetriever`	Dense + sparse hybrid search via native Reciprocal Rank Fusion (RRF)

Installation

pip install upstash-haystack

Usage

To use Upstash Vector as your data storage for Haystack LLM pipelines, you must have an Upstash account and a Vector index. Once you have those, set your credentials as environment variables:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Valkey is a high-performance, in-memory data structure store that you can use in Haystack pipelines with the ValkeyDocumentStore. Valkey operates in-memory by default for maximum performance, but can be configured with persistence options for data durability.

For a detailed overview of all the available methods and settings for the ValkeyDocumentStore, visit the Haystack API Reference.

Installation

pip install valkey-haystack

Usage

To use Valkey as your data storage for your Haystack LLM pipelines, you must have a Valkey server with search module running. Learn how to spin up a Valkey server in the Running Valkey-Haystack Locally section. Once you have that, you can initialize a ValkeyDocumentStore for Haystack:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Haystack components for integrating Valyu’s powerful search and content extraction APIs into your Haystack pipelines.

This package provides two main components:

ValyuSearch - Search component that queries the Valyu DeepSearch API and returns documents with content already included
ValyuContentFetcher - Content extraction component that fetches and cleans content from URLs

Key Features:

Search across web and proprietary sources
Full content included in search results
AI-powered content extraction and summarization

Installation

Use pip to install Valyu Search for Haystack:

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Vespa is an open-source search engine and vector database that supports vector search, lexical search, and search in structured data, all in the same query. This integration lets you use Vespa as a DocumentStore in Haystack pipelines and provides retrievers for both embedding-based and keyword-based search.

It is built on top of pyvespa and expects a Vespa application to be running and reachable (locally via Docker, on Vespa Cloud, or self-hosted). The Vespa schema, including the fields and ranking profiles used by the retrievers, must be defined on the Vespa application before you start indexing or querying.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs. It is an open-source project that allows serving open models in production, when you have GPU resources available.

vLLM serves models behind an OpenAI-compatible HTTP server and supports generative, embedding, and ranking models. The vllm-haystack integration provides dedicated Haystack components that connect to a running vLLM server.

Installation

Install vLLM following the official instructions. For production use cases, there are other options, including Docker.

Mon, 01 Jan 0001 00:00:00 +0000

Voyage AI’s embedding and ranking models are state-of-the-art in retrieval accuracy. The integration supports the following models:

voyage-4-large, voyage-4, and voyage-4-lite - Latest general-purpose embedding models with shared embedding space and MoE architecture
voyage-3.5 and voyage-3.5-lite - General-purpose embedding models with superior performance
voyage-code-3 - Optimized for code retrieval
voyage-context-3 - Contextualized chunk embedding model that preserves document context for improved retrieval accuracy
voyage-multimodal-3.5 - Multimodal model supporting text, images, and video (preview)

For the complete list of available models, see the Embeddings Documentation and Contextualized Chunk Embeddings.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

IBM watsonx.ai provides access to IBM’s foundation models for enterprise AI. This integration allows you to use powerful models like granite-3-2b-instruct and slate-125m-english-rtrvr with Haystack for chat and embedding tasks.

Installation

Install the IBM Watsonx integration:

pip install watsonx-haystack

Usage

Once installed, you will have access to the Haystack components:

WatsonxChatGenerator: Use this component with IBM watsonx models like granite-3-2b-instruct for chat generation.
WatsonxGenerator: Use this component with IBM watsonx models like granite-3-2b-instruct for simple text generation tasks.
WatsonxDocumentEmbedder: Use this component with IBM watsonx models like slate-125m-english-rtrvr for generating document embeddings.
WatsonxTextEmbedder: Use this component with IBM watsonx models like slate-125m-english-rtrvr for generating text embeddings and retrieval.

To use the Watsonx integration, you must provide your WATSONX_API_KEY and WATSONX_PROJECT_ID via environment variables or as an init argument. If neither is set, you won’t be able to use the components.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Installation

Use pip to install Weaviate:

pip install weaviate-haystack

Usage

Once installed, initialize your Weaviate database to use it with Haystack.

In this example, we use the temporary embedded version for simplicity. To use a self-hosted Docker container or Weaviate Cloud Service, take a look at the docs.

from haystack_integrations.document_stores.weaviate import WeaviateDocumentStore
from weaviate.embedded import EmbeddedOptions

document_store = WeaviateDocumentStore(embedded_options=EmbeddedOptions())
# document_store = WeaviateDocumentStore(url="http://localhost:8080")

Writing Documents to WeaviateDocumentStore

To write documents to WeaviateDocumentStore, create an indexing pipeline.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This integration allows you to use Weights & Biases Weave framework for tracing and monitoring Haystack pipeline components. It provides a connector that sends Haystack traces to Weights & Biases for monitoring and visualization.

Installation

pip install weave-haystack

Usage

Components

This integration introduces one new component, a connector named WeaveConnector whose only responsibility is to send traces to Weights & Biases.

Note that you need to have the WANDB_API_KEY environment variable set to your Weights & Biases API key.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The whisper-haystack integration provides two components that transcribe audio files into Haystack documents using OpenAI’s Whisper model:

LocalWhisperTranscriber: runs Whisper on your own machine. The audio is never sent to a third party.
RemoteWhisperTranscriber: transcribes audio with the OpenAI Whisper API (and other OpenAI-compatible providers).

Both components are typically used as the first step of an indexing pipeline. They were previously part of Haystack core and now live in the whisper-haystack integration package, maintained in haystack-core-integrations.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

The xquik-haystack package provides Haystack web search components for public X/Twitter data through the Xquik REST API.

It includes:

XquikTweetSearch: searches public posts and returns Haystack Document objects.
XquikUserTweetsFetcher: fetches recent public posts for an X user and returns Haystack Document objects.

Both components return documents, links, has_more, and next_cursor, making them usable in standalone retrieval steps or larger Haystack pipelines.

Installation

pip install xquik-haystack

Create an API key from the Xquik dashboard, or follow the Xquik quickstart for account and API-key setup.

Mon, 01 Jan 0001 00:00:00 +0000

Overview

You.com provides a Search API built for AI agents and applications, returning clean, structured web and news results.

This integration provides:

YouComWebSearch: Searches the web using the You.com Search API and returns results as Haystack Document objects along with source URLs.

YouComWebSearch works with zero configuration: when no API key is available, it searches using You.com’s keyless free tier (rate limited per IP), so getting-started pipelines run without any setup. Set the YOUDOTCOM_API_KEY environment variable to use the keyed You.com Search API with higher limits. You can get a free API key at you.com/platform.

Community

Mon, 01 Jan 0001 00:00:00 +0000

Dakshaa Mavathur

Mon, 01 Jan 0001 00:00:00 +0000

Hacktoberfest 2023

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Haystack Benchmarks

Mon, 01 Jan 0001 00:00:00 +0000

Kader Miyanyedi

Mon, 01 Jan 0001 00:00:00 +0000

Nicola Procopio

Mon, 01 Jan 0001 00:00:00 +0000

NLP Resources

Mon, 01 Jan 0001 00:00:00 +0000

Here are some links to resources about the core concepts of Natural Language Processing (NLP) that will help you get started with Haystack.

What is NLP?

Learn about what is possible when we apply computational power to language processing.

Title	Type	Author	Description	Level
Natural Language Processing (NLP)	Blog	IBM	High level introduction to the tasks, tools, and use cases of NLP.	Beginner
Introduction to NLP	Video	Data Science Dojo	Covers many of the different tasks from part-of-speech tagging to the creation of word embeddings. Contains some probabilistic notation.	Intermediate
Text Classification with NLP: Tf-Idf vs Word2Vec vs BERT	Blog with Code	Mauro Di Pietro	Hands-on and in depth dive into text classification using TF-IDF, Word2Vec and BERT.	Intermediate

Search and Question Answering

There are many different flavors of search. Learn the differences between them and understand how the task of question answering can improve the search experience.

Spring into Haystack

Mon, 01 Jan 0001 00:00:00 +0000

🌱 Sprout an Agent with Haystack & MCP

Spring has sprung — flowers are blooming, birds are chirping, and it’s the perfect time for some fresh ideas to take root 🐥

Spring cleaning can wait… you’re here to level up your AI dev skills with Haystack and take part in a blossoming new movement: Anthropic’s Model Context Protocol (MCP) 🌐

This challenge invites you to sprout a Haystack Agent that speaks MCP and connects to external systems — starting with GitHub.

Sriniketh Jayasendil

Mon, 01 Jan 0001 00:00:00 +0000

Syed Shahmeer Ali

Mon, 01 Jan 0001 00:00:00 +0000

Tarun Jain

Mon, 01 Jan 0001 00:00:00 +0000