Tutorial: Creating a Multi-Agent System with Haystack

_{Last Updated:
June 23, 2025}

Level: Advanced
Time to complete: 20 minutes
Components Used: Agent, DuckduckgoApiWebSearch, OpenAIChatGenerator, DocumentWriter
Prerequisites: You need an OpenAI API Key, and a Notion Integration set up beforehand
Goal: After completing this tutorial, you’ll have learned how to build a multi-agent system in Haystack where each agent is specialized for a specific task.

Overview

Multi-agent systems are made up of several intelligent agents that work together to solve complex tasks more effectively than a single agent alone. Each agent takes on a specific role or skill, allowing for distributed reasoning, task specialization, and smooth coordination within one unified system.

What makes this possible in Haystack is the ability to use agents as tools for other agents. This powerful pattern allows you to compose modular, specialized agents and orchestrate them through a main agent that delegates tasks based on context.

In this tutorial, you’ll build a simple yet powerful multi-agent setup with one main agent and two sub-agents: one focused on researching information, and the other on saving it.

Preparing the Environment

First, let’s install required packages:

%%bash

pip install -q haystack-ai duckduckgo-api-haystack

Enter API Keys

Enter API keys required for this tutorial. Learn how to get your NOTION_API_KEY after creating a Notion integration here

from getpass import getpass
import os

if not os.environ.get("OPENAI_API_KEY"):
    os.environ["OPENAI_API_KEY"] = getpass("Enter your OpenAI API key:")
if not os.environ.get("NOTION_API_KEY"):
    os.environ["NOTION_API_KEY"] = getpass("Enter your NOTION API key:")

Creating Tools for the Research Agent

Let’s set up the tools for your research agent. Its job is to gather information on a topic, and in this tutorial, it will use just two sources: the whole web and Wikipedia. In other scenarios, it could also connect to a retrieval or RAG pipeline linked to a document store.

You’ll create two ComponentTools using the DuckduckgoApiWebSearch component: one for general web search, and one limited to Wikipedia by setting allowed_domain.

One more step before wrapping up: DuckduckgoApiWebSearch returns a list of documents, but the Agent works best when tools return a single string in this setting. To handle that, define a doc_to_string function that converts the Document list into a string. This function, used as the outputs_to_string handler, can also add custom elements like filenames or links before returning the output.

from haystack.tools import ComponentTool
from duckduckgo_api_haystack import DuckduckgoApiWebSearch


def doc_to_string(documents) -> str:
    """
    Handles the tool output before conversion to ChatMessage.
    """
    result_str = ""
    for document in documents:
        result_str += f"File Content for {document.meta['link']}\n\n {document.content}"

    if len(result_str) > 150_000:  # trim if the content is too large
        result_str = result_str[:150_000] + "...(large file can't be fully displayed)"

    return result_str


web_search = ComponentTool(
    component=DuckduckgoApiWebSearch(top_k=5, backend="lite"),
    name="web_search",
    description="Search the web",
    outputs_to_string={"source": "documents", "handler": doc_to_string},
)

wiki_search = ComponentTool(
    component=DuckduckgoApiWebSearch(top_k=5, backend="lite", allowed_domain="https://en.wikipedia.org"),
    name="wiki_search",
    description="Search Wikipedia",
    outputs_to_string={"source": "documents", "handler": doc_to_string},
)

If you’re hitting rate limits with DuckDuckGo, you can use SerperDevWebSearch as your websearch component for these tools. You need to enter the free Serper API Key to use SerperDevWebSearch.

# from getpass import getpass
# import os

# from haystack.components.websearch import SerperDevWebSearch
# from haystack.tools import ComponentTool

# if not os.environ.get("SERPERDEV_API_KEY"):
#     os.environ["SERPERDEV_API_KEY"] = getpass("Enter your SERPER API key:")

# web_search = ComponentTool(
#     component=SerperDevWebSearch(top_k=5),
#     name="web_search",
#     description="Search the web",
#     outputs_to_string={"source": "documents", "handler": doc_to_string},
# )

# wiki_search = ComponentTool(
#     component=SerperDevWebSearch(top_k=5, allowed_domains=["https://www.wikipedia.org/", "https://en.wikipedia.org"]),
#     name="wiki_search",
#     description="Search Wikipedia",
#     outputs_to_string={"source": "documents", "handler": doc_to_string},
# )

Initializing the Research Agent

Now it’s time to bring your research agent to life. This agent will solely responsible for finding information. Use OpenAIChatGenerator or any other chat generator that supports function calling.

Pass in the web_search and wiki_search tools you created earlier. To keep things transparent, enable streaming using the built-in print_streaming_chunk function. This will display the agent’s tool calls and results in real time, so you can follow its actions step by step.

from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.generators.chat import OpenAIChatGenerator

research_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="""
    You are a research agent that can find information on web or specifically on wikipedia.
    Use wiki_search tool if you need facts and use web_search tool for latest news on topics.
    Use one tool at a time. Try different queries if you need more information.
    Only use the retrieved context, do not use your own knowledge.
    Summarize the all retrieved information before returning response to the user.
    """,
    tools=[web_search, wiki_search],
    streaming_callback=print_streaming_chunk,
)

result = research_agent.run(
    messages=[ChatMessage.from_user("Can you tell me about Florence Nightingale's contributions to nursery?")]
)

Print the final answer of the agent through result["last_message"].text

print("Final Answer:", result["last_message"].text)

Creating Tools for the Writer Agent

Next, let’s set up tools for your second agent, the writer agent. Its job is to save content, either to Notion or to a document store (to InMemoryDocumentStore in this tutorial).

Notion Writer Tool

Start by creating a custom component that can add a new page to a Notion workspace given the page title and content. To use it, you’ll need an active Notion integration with access to a parent page where new content will be stored. Grab the page_id of that parent page from its URL, and pass it as an init parameter when initializing the component.

Here’s a basic implementation to get you started:

from haystack import component
from typing import Optional
from haystack.utils import Secret
import requests


@component
class NotionPageCreator:
    """
    Create a page in Notion using provided title and content.
    """

    def __init__(
        self,
        page_id: str,
        notion_version: str = "2022-06-28",
        api_key: Secret = Secret.from_env_var("NOTION_API_KEY"),  # to use the environment variable NOTION_API_KEY
    ):
        """
        Initialize with the target Notion database ID and API version.
        """
        self.api_key = api_key
        self.notion_version = notion_version
        self.page_id = page_id

    @component.output_types(success=bool, status_code=int, error=Optional[str])
    def run(self, title: str, content: str):
        """
        :param title: The title of the Notion page.
        :param content: The content of the Notion page.
        """
        headers = {
            "Authorization": f"Bearer {self.api_key.resolve_value()}",
            "Content-Type": "application/json",
            "Notion-Version": self.notion_version,
        }

        payload = {
            "parent": {"page_id": self.page_id},
            "properties": {"title": [{"text": {"content": title}}]},
            "children": [
                {
                    "object": "block",
                    "type": "paragraph",
                    "paragraph": {"rich_text": [{"type": "text", "text": {"content": content}}]},
                }
            ],
        }

        response = requests.post("https://api.notion.com/v1/pages", headers=headers, json=payload)

        if response.status_code == 200 or response.status_code == 201:
            return {"success": True, "status_code": response.status_code}
        else:
            return {"success": False, "status_code": response.status_code, "error": response.text}

Give the component a quick test to confirm it’s working properly.

notion_writer = NotionPageCreator(page_id="<your_page_id>")
notion_writer.run(title="My first page", content="The content of my first page")

💡 When turning a custom component into a tool using ComponentTool, make sure its input parameters are well-defined. You can do this in one of two ways:

Pass a properties dictionary to ComponentTool, or
Use parameter annotations in the run method’s docstring, like so:

    def run(self, title: str, content: str):
        """
        :param title: The title of the Notion page.
        :param content: The content of the Notion page.
        """

This approach also applies to setting the tool’s description.

from haystack.tools import ComponentTool

notion_writer = ComponentTool(
    component=NotionPageCreator(page_id="<your_page_id>"),
    name="notion_writer",
    description="Use this tool to write/save content to Notion.",
)
notion_writer.parameters  # see how parameters are automatically generated by the ComponentTool

Document Store Writer Tool

Let’s now build the other tool for the writer agent, this one will save content to an InMemoryDocumentStore.

To make this work, start by creating a pipeline that includes the custom DocumentAdapter compoenent along with the DocumentWriter. Once the pipeline is ready, wrap it in a SuperComponent and then convert it into a tool using ComponentTool.

💡 Tip: You could also create a tool from a simple function that runs the pipeline. However, the recommended approach is to use SuperComponent together with ComponentTool, especially if you plan to deploy the tool with Hayhooks, since this method supports better serialization. Learn more about SuperComponents in Tutorial: Creating Custom SuperComponents

from haystack import Pipeline, component, Document, SuperComponent
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from typing import List


@component
class DocumentAdapter:
    @component.output_types(documents=List[Document])
    def run(self, content: str, title: str):
        return {"documents": [Document(content=content, meta={"title": title})]}


document_store = InMemoryDocumentStore()

doc_store_writer_pipeline = Pipeline()
doc_store_writer_pipeline.add_component("adapter", DocumentAdapter())
doc_store_writer_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
doc_store_writer_pipeline.connect("adapter", "writer")

doc_store_writer = ComponentTool(
    component=SuperComponent(doc_store_writer_pipeline),
    name="doc_store_writer",
    description="Use this tool to write/save content to document store",
    parameters={
        "type": "object",
        "properties": {
            "title": {"type": "string", "description": "The title of the Document"},
            "content": {"type": "string", "description": "The content of the Document"},
        },
        "required": ["title", "content"],
    },
)
doc_store_writer.parameters

Initializing the Writer Agent

Now let’s bring the writer agent to life. Its job is to save or write information using the tools you’ve set up.

Provide the notion_writer and doc_writer tools as inputs. In other use cases, you could include tools for other platforms like Google Drive, or connect to MCP servers using MCPTool.

Enable streaming with the built-in print_streaming_chunk function to see the agent’s actions in real time. Be sure to write a clear and descriptive system prompt to guide the agent’s behavior.

Lastly, set exit_conditions=["notion_writer", "doc_writer"] so the agent knows to stop once it calls one of these tools. Since this agent’s job is to act, not reply, we don’t want it to return a final response.

from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage
from haystack.components.generators.utils import print_streaming_chunk
from haystack.components.generators.chat import OpenAIChatGenerator

writer_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="""
    You are a writer agent that saves given information to different locations.
    Do not change the provided content before saving.
    Infer the title from the text if not provided.
    When you need to save provided information to Notion, use notion_writer tool.
    When you need to save provided information to document store, use doc_store_writer tool
    If no location is mentioned, use notion_writer tool to save the information.
    """,
    tools=[doc_store_writer, notion_writer],
    streaming_callback=print_streaming_chunk,
    exit_conditions=["notion_writer", "doc_store_writer"],
)

Let’s test the Writer Agent

result = writer_agent.run(
    messages=[
        ChatMessage.from_user(
            """
Save this text on Notion:

Florence Nightingale is widely recognized as the founder of modern nursing, and her contributions significantly transformed the field.
Florence Nightingale's legacy endures, as she set professional standards that have shaped nursing into a respected and essential component of the healthcare system. Her influence is still felt in nursing education and practice today.
"""
        )
    ]
)

Creating the Multi-Agent System

So far, you’ve built two sub-agents, one for research and one for writing, along with their respective tools. Now it’s time to bring everything together into a single multi-agent system.

To do this, wrap both research_agent and writer_agent with ComponentTool, then pass them as tools to your main_agent. This setup allows the main agent to coordinate the overall workflow by delegating tasks to the right sub-agent, each of which already knows how to handle its own tools.

from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.generators.utils import print_streaming_chunk

research_tool = ComponentTool(
    component=research_agent,
    description="Use this tool to find information on web or specifically on wikipedia",
    name="research_tool",
    outputs_to_string={"source": "last_message"},
)
writer_tool = ComponentTool(
    component=writer_agent,
    description="Use this tool to write content into document store or Notion",
    name="writer_tool",
    outputs_to_string={"source": "last_message"},
)

main_agent = Agent(
    chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
    system_prompt="""
    You are an assistant that has access to several tools.
    Understand the user query and use relevant tool to answer the query.
    You can use `research_tool` to make research on web and wikipedia and `writer_tool` to save information into the document store or Notion.
    """,
    streaming_callback=print_streaming_chunk,
    tools=[research_tool, writer_tool],
)

Let’s test this multi-agent system!

from haystack.dataclasses import ChatMessage

result = main_agent.run(
    messages=[
        ChatMessage.from_user(
            """
            Can you research the history of the Silk Road?
            """
        )
    ]
)

result["last_message"].text

result = main_agent.run(
    messages=[
        ChatMessage.from_user(
            """
            Summarize how RAG pipelines work and save it in Notion
            """
        )
    ]
)

What’s next

🎉 Congratulations! You’ve just built a multi-agent system with Haystack, where specialized agents work together to research and write, each with their own tools and responsibilities. You now have a flexible foundation for building more complex, modular agent workflows.

Curious to keep exploring? Here are a few great next steps:

To stay up to date on the latest Haystack developments, you can sign up for our newsletter or join Haystack discord community.

Query Classification with TransformersTextRouter and TransformersZeroShotTextRouter

Building a Chat Agent with Function Calling