{ "cells": [ { "cell_type": "markdown", "id": "10827537", "metadata": {}, "source": [ "# Tutorial: Using Pre-Built Agents from Agent Pack\n", "\n", "- **Level**: Advanced\n", "- **Time to complete**: 25 minutes\n", "- **Components/Packages Used**: [`agent-pack-haystack`](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack) ([`create_advanced_rag_agent`](https://docs.haystack.deepset.ai/docs/advanced-rag-agent), [`create_deep_research_agent`](https://docs.haystack.deepset.ai/docs/deep-research-agent)), [`Agent`](https://docs.haystack.deepset.ai/docs/agent), [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore)\n", "- **Prerequisites**: Haystack 3.0 or later, an [OpenAI API key](https://platform.openai.com/api-keys), and a [Tavily API key](https://app.tavily.com) (free tier available)\n", "- **Goal**: After completing this tutorial, you'll understand what Agent Pack is and why it exists, and you'll have run and customized two ready-made agents: the **Advanced RAG Agent** and the **Deep Research Agent**." ] }, { "cell_type": "markdown", "id": "98dd9cf3", "metadata": {}, "source": [ "## Overview\n", "\n", "A language model and a set of tools are the core building blocks of an agent. But a *robust* agent usually needs more than that: an architecture that splits work across sub-agents, techniques for keeping the context window small but focused, and hooks to shape the agent loop before and after it runs. Assembling those pieces correctly is where most of the effort goes.\n", "\n", "[**Agent Pack**](https://docs.haystack.deepset.ai/docs/agent-pack) is a collection of complex, pre-configured Haystack agents that package these techniques into working agents. Each one is a complete architecture built from Haystack primitives ([`Agent`](https://docs.haystack.deepset.ai/docs/agent), [Tools](https://docs.haystack.deepset.ai/docs/tool), [hooks](https://docs.haystack.deepset.ai/docs/hooks), [`State`](https://docs.haystack.deepset.ai/docs/state), and Pipelines), exposed behind a single `create_*` entry point.\n", "\n", "There are three ways to use an agent from the pack:\n", "\n", "- **Run it as is.** Call the `create_*` function, pass a question, and get an answer. The defaults are chosen to work out of the box.\n", "- **Customize it.** Each entry point exposes keyword arguments for swapping models, adding tools, and tuning behavior.\n", "- **Copy it.** Read the implementation and adapt it as a blueprint for your own architecture.\n", "\n", "### Why these two agents?\n", "\n", "The pack ships agents that solve genuinely hard, recurring problems, so you don't have to rebuild the architecture each time:\n", "\n", "- The **Advanced RAG Agent** tackles a common RAG failure mode: an agent that *guesses* which metadata fields and values exist and builds broken filters. Instead, this agent inspects the document store (fields, values, ranges) and constructs valid Haystack filters to narrow retrieval. Reach for it when your corpus has rich metadata (categories, dates, ratings, languages...) and answering questions well depends on filtering by it.\n", "- The **Deep Research Agent** is a canonical multi-agent pattern: it researches a question on the web with an orchestrator that delegates focused sub-questions to isolated sub-researchers, then writes a cited Markdown report. Its key idea is **context management** through isolation and compression. Reach for it when a question needs broad, multi-source web research rather than a single lookup.\n", "\n", "In this tutorial you'll run both agents, look at what they return, and customize them." ] }, { "cell_type": "markdown", "id": "a1114b46", "metadata": {}, "source": "> ⚠️ **Agent Pack is experimental.** Its APIs and agent architectures can change in any release, without following the usual Haystack deprecation policy. It is distributed separately from `haystack-ai` (in [`haystack-core-integrations`](https://github.com/deepset-ai/haystack-core-integrations)) precisely so these patterns can evolve quickly. Expect things to change and occasionally break, and pin a version if you depend on it in production." }, { "cell_type": "markdown", "id": "0549fb8d", "metadata": {}, "source": "## Preparing the Environment\n\nFirst, install `agent-pack-haystack` along with the optional runtime dependencies the two agents need:\n\n- `arrow` renders today's date into the agents' system prompts (both the Advanced RAG and Deep Research agents use it), so they can reason about relative dates like \"the last 5 years\".\n- `tavily-haystack`, `trafilatura`, and `pypdf` power the Deep Research Agent's web search and its HTML/PDF page reading." }, { "cell_type": "code", "execution_count": null, "id": "92f47f96", "metadata": {}, "outputs": [], "source": [ "%%bash\n", "\n", "pip install -q agent-pack-haystack arrow tavily-haystack trafilatura pypdf" ] }, { "cell_type": "markdown", "id": "26c24d26", "metadata": {}, "source": [ "### Enter API Keys\n", "\n", "Both agents use OpenAI models by default. The Deep Research Agent additionally uses [Tavily](https://tavily.com) for web search. Enter the keys below:" ] }, { "cell_type": "code", "execution_count": null, "id": "87b8157b", "metadata": {}, "outputs": [], "source": [ "from getpass import getpass\n", "import os\n", "\n", "if not os.environ.get(\"OPENAI_API_KEY\"):\n", " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key:\")\n", "if not os.environ.get(\"TAVILY_API_KEY\"):\n", " os.environ[\"TAVILY_API_KEY\"] = getpass(\"Enter your Tavily API key:\")" ] }, { "cell_type": "markdown", "id": "767401d6", "metadata": {}, "source": [ "## Part 1: The Advanced RAG Agent\n", "\n", "The Advanced RAG Agent answers questions from documents it retrieves out of a document store. What makes it *advanced* is that it doesn't guess your metadata: it can inspect the store, discover which fields and values exist, and build a [metadata filter](https://docs.haystack.deepset.ai/docs/metadata-filtering) to narrow retrieval when that helps. Plain, unfiltered retrieval stays available when it doesn't.\n", "\n", "### Indexing a corpus with metadata\n", "\n", "Let's index a small set of documents with varied metadata (`category`, `year`, `rating`), then ask a question that can only be answered well by filtering on that metadata." ] }, { "cell_type": "code", "execution_count": null, "id": "4bda1e15", "metadata": {}, "outputs": [], "source": [ "from haystack import Document\n", "from haystack.document_stores.in_memory import InMemoryDocumentStore\n", "\n", "document_store = InMemoryDocumentStore()\n", "document_store.write_documents(\n", " [\n", " Document(\n", " content=\"CRISPR gene editing corrected a hereditary blindness mutation in a clinical trial.\",\n", " meta={\"category\": \"science\", \"year\": 2021, \"rating\": 4.6},\n", " ),\n", " Document(\n", " content=\"A quantum computer demonstrated error-corrected logical qubits.\",\n", " meta={\"category\": \"science\", \"year\": 2023, \"rating\": 4.8},\n", " ),\n", " Document(\n", " content=\"Dolly the sheep became the first mammal cloned from an adult somatic cell.\",\n", " meta={\"category\": \"science\", \"year\": 1996, \"rating\": 4.2},\n", " ),\n", " Document(\n", " content=\"The Berlin Wall fell, a decisive moment in the end of the Cold War.\",\n", " meta={\"category\": \"history\", \"year\": 1989, \"rating\": 4.7},\n", " ),\n", " Document(\n", " content=\"Argentina won the FIFA World Cup final against France on penalties.\",\n", " meta={\"category\": \"sports\", \"year\": 2022, \"rating\": 4.9},\n", " ),\n", " ]\n", ")" ] }, { "cell_type": "markdown", "id": "d4fa6802", "metadata": {}, "source": [ "### Creating the agent\n", "\n", "`create_advanced_rag_agent` needs two things: the `document_store` (which feeds the metadata-inspection tools and a direct fetch-by-filter tool) and a `retriever` (which becomes the agent's relevance-scoring `search_documents` tool). Here we use a keyword [`InMemoryBM25Retriever`](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever), but it can also be an embedding retriever or a full retrieval `Pipeline`.\n", "\n", "The retriever you pass should be **relevance-scoring** (BM25, embedding, or hybrid). Direct, unscored fetching by metadata is already handled by the agent's built-in `fetch_documents_by_filter` tool." ] }, { "cell_type": "code", "execution_count": null, "id": "919aa4e3", "metadata": {}, "outputs": [], "source": [ "from haystack.components.retrievers.in_memory import InMemoryBM25Retriever\n", "from haystack_integrations.agent_pack import create_advanced_rag_agent\n", "\n", "rag_agent = create_advanced_rag_agent(\n", " document_store=document_store, retriever=InMemoryBM25Retriever(document_store=document_store, top_k=5)\n", ")" ] }, { "cell_type": "markdown", "id": "3b5c2c9a", "metadata": {}, "source": [ "### Running the agent\n", "\n", "Call the agent with your question as a user message. We pass [`print_streaming_chunk`](https://docs.haystack.deepset.ai/docs/agent#streaming) as the `streaming_callback` so you can watch the agent work: it first lists the metadata fields, checks the `category` values and the `year` range, then retrieves with a filter it builds itself." ] }, { "cell_type": "code", "execution_count": null, "id": "e9a5b1e8", "metadata": {}, "outputs": [], "source": [ "from haystack.dataclasses import ChatMessage\n", "from haystack.components.generators.utils import print_streaming_chunk\n", "\n", "result = rag_agent.run(\n", " messages=[ChatMessage.from_user(\"What science advances happened after 2015?\")],\n", " streaming_callback=print_streaming_chunk,\n", ")" ] }, { "cell_type": "markdown", "id": "68cada14", "metadata": {}, "source": [ "The agent's answer is in `last_message`, and every document it retrieved during the run is accumulated (deduplicated by id) under `documents`. The answer cites documents by the first 8 characters of their id, for example `[doc a1b2c3d4]`." ] }, { "cell_type": "code", "execution_count": null, "id": "9d7c51b5", "metadata": {}, "outputs": [], "source": [ "print(\"ANSWER:\\n\", result[\"last_message\"].text)\n", "\n", "print(\"\\nRETRIEVED DOCUMENTS:\")\n", "for doc in result[\"documents\"]:\n", " print(f\"[doc {doc.id[:8]}] {doc.meta} :: {doc.content[:60]}\")" ] }, { "cell_type": "markdown", "id": "f48421c0", "metadata": {}, "source": [ "### How it works under the hood\n", "\n", "The agent is a single Haystack [`Agent`](https://docs.haystack.deepset.ai/docs/agent) that works through three logical stages using five tools:\n", "\n", "- **Inspect metadata** with `list_metadata_fields`, `get_metadata_field_values`, and `get_metadata_field_range` (the system prompt tells it to list the fields first).\n", "- **Retrieve documents** with `search_documents` (your relevance-scoring retriever, optionally narrowed by a filter) or `fetch_documents_by_filter` (a direct metadata lookup when scoring isn't needed).\n", "- **Answer** using only the retrieved documents, citing each as `[doc ]`.\n", "\n", "To keep filters valid, the Haystack filter grammar is embedded directly in the retrieval tools' parameter descriptions, so the model learns it at the point of use rather than from a long system prompt. Every retrieved document is accumulated in the agent's [`State`](https://docs.haystack.deepset.ai/docs/state) under `documents` (deduplicated by id), which is why the run returns them alongside the answer. And if the loop is cut off by `max_agent_steps` before an answer is written, a built-in `BackupAnswerHook` (an `after_run` hook) makes one extra call to produce a best-effort answer, so `last_message` always carries text.\n", "\n", "> 💡 The metadata tools rely on document store methods (`get_metadata_fields_info`, `get_metadata_field_unique_values`, `get_metadata_field_min_max`) that aren't part of the base `DocumentStore` protocol. `InMemoryDocumentStore` and most integrations implement them (OpenSearch, Elasticsearch, Weaviate, Chroma, pgvector, Qdrant, Pinecone, and more)." ] }, { "cell_type": "markdown", "id": "dcd8b9e2", "metadata": {}, "source": [ "### Customizing the agent\n", "\n", "Everything is configured through keyword arguments. Only `document_store` and `retriever` are required. A few useful ones:\n", "\n", "- `llm`: the chat generator that drives the agent loop. Defaults to `OpenAIResponsesChatGenerator(\"gpt-5.4\")` with low reasoning effort. Swap in any tool-calling generator, from OpenAI or another provider.\n", "- `max_agent_steps`: caps the loop (default `20`). If the loop is cut off before an answer is written, a built-in `BackupAnswerHook` makes one extra call to produce a best-effort answer, so `last_message` always carries text.\n", "- `max_fetched_docs`: how many documents `fetch_documents_by_filter` shows per call (default `10`).\n", "- `extra_tools`, `state_schema`, `hooks`: extend the agent with your own tools, state, and hooks.\n", "\n", "Here we swap the LLM for a smaller, widely available model and tighten the step budget:" ] }, { "cell_type": "code", "execution_count": null, "id": "edee54d4", "metadata": {}, "outputs": [], "source": [ "from haystack.components.generators.chat import OpenAIChatGenerator\n", "\n", "custom_rag_agent = create_advanced_rag_agent(\n", " document_store=document_store,\n", " retriever=InMemoryBM25Retriever(document_store=document_store, top_k=5),\n", " llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " max_agent_steps=10,\n", ")\n", "\n", "result = custom_rag_agent.run(messages=[ChatMessage.from_user(\"Which documents describe events before 1990?\")])\n", "print(result[\"last_message\"].text)" ] }, { "cell_type": "markdown", "id": "41b3360b", "metadata": {}, "source": [ "**Using a retrieval pipeline.** To use a multi-component retrieval flow (for example hybrid retrieval), pass a `Pipeline` as `retriever` and supply `retrieval_pipeline_input_mapping` (mapping the tool's `query` and `filters` to your pipeline's input sockets) and, optionally, `retrieval_pipeline_output_mapping`. See the [Advanced RAG Agent docs](https://docs.haystack.deepset.ai/docs/advanced-rag-agent) for a full hybrid-retrieval example.\n", "\n", "**Using the tools on their own.** The four document-store-backed tools are exported individually and bundled as `DocumentStoreToolset`, so you can drop them into your own `Agent` with your own prompt, treating the pack as a toolbox rather than a finished agent:\n", "\n", "```python\n", "from haystack.components.agents import Agent\n", "from haystack_integrations.agent_pack.advanced_rag import DocumentStoreToolset\n", "\n", "agent = Agent(\n", " chat_generator=...,\n", " tools=[DocumentStoreToolset(document_store), my_retrieval_tool],\n", ")\n", "```" ] }, { "cell_type": "markdown", "id": "8efe9b0b", "metadata": {}, "source": [ "## Part 2: The Deep Research Agent\n", "\n", "The Deep Research Agent takes a question, researches it on the web, and produces a structured, cited Markdown report. Under the hood it's a small multi-agent system (we'll look at how it works right after running it), but the entry point is a single call." ] }, { "cell_type": "markdown", "id": "1f20f77e", "metadata": {}, "source": [ "### Running the agent\n", "\n", "`create_deep_research_agent()` works with zero arguments. But the defaults research broadly (up to 5 sub-questions, 5 concurrent sub-researchers, 20 steps each), which can take several minutes and cost real tokens. For this tutorial we'll deliberately scope it down, which also previews the customization knobs. Even so, expect this cell to take a couple of minutes." ] }, { "cell_type": "code", "execution_count": null, "id": "d55db360", "metadata": {}, "outputs": [], "source": [ "from haystack_integrations.agent_pack import create_deep_research_agent\n", "\n", "research_agent = create_deep_research_agent(\n", " max_subtopics=2, # delegate at most 2 sub-questions (breadth)\n", " max_concurrent_researchers=2, # run at most 2 sub-researchers at once\n", " max_researcher_steps=6, # cap each sub-researcher's search/read/think loop\n", " max_search_results=5, # results per web_search call\n", ")" ] }, { "cell_type": "code", "execution_count": null, "id": "8dc042f6", "metadata": {}, "outputs": [], "source": [ "from haystack.dataclasses import ChatMessage\n", "\n", "result = research_agent.run(\n", " messages=[ChatMessage.from_user(\"What are the main techniques for managing the context window in LLM agents?\")]\n", ")\n", "\n", "print(result[\"report\"])" ] }, { "cell_type": "markdown", "id": "586d0d9e", "metadata": {}, "source": [ "The main output is `report`: the final Markdown report with inline `[text](url)` citations. The run also returns the intermediate `brief` and `notes`, plus the standard `Agent` outputs (`messages`, `last_message`, `step_count`, `token_usage`, `tool_call_counts`). Let's look at the brief the Scope phase produced and how many research notes were collected:" ] }, { "cell_type": "code", "execution_count": null, "id": "5e88ffe8", "metadata": {}, "outputs": [], "source": [ "print(\"RESEARCH BRIEF:\\n\", result[\"brief\"])\n", "print(f\"\\nCollected {len(result['notes'])} research note(s).\")\n", "print(\"\\nFIRST NOTE (a sub-researcher summary):\\n\", result[\"notes\"][0][:800])" ] }, { "cell_type": "markdown", "id": "39aabb3e", "metadata": {}, "source": [ "### How it works under the hood\n", "\n", "The agent is built around a single top-level [`Agent`](https://docs.haystack.deepset.ai/docs/agent) acting as the **orchestrator**, with two [hooks](https://docs.haystack.deepset.ai/docs/hooks) that wrap its loop to create three phases:\n", "\n", "- **Scope** (a `before_run` hook): a plain LLM call rewrites your question into a focused research `brief`, stored on the agent's [`State`](https://docs.haystack.deepset.ai/docs/state).\n", "- **Research** (the orchestrator loop): the orchestrator splits the brief into a few non-overlapping sub-questions and delegates each to an isolated **sub-researcher** agent, exposed to it as a tool. It can fire several off at once (bounded by `max_concurrent_researchers`) and collects their summaries into a shared `notes` list.\n", "- **Write** (an `after_run` hook): a final LLM call turns the brief plus `notes` into the `report`.\n", "\n", "Each **sub-researcher** is its own `Agent` that searches the web (`web_search`), optionally reads promising pages (`read_url`, which fetches and summarizes a page or PDF toward the question), reflects (`think_tool`), and finishes by writing one short, cited summary.\n", "\n", "The key idea is **context management**. Raw web content (search results, full pages, PDFs) is large and noisy; if it all piled into one context window, output quality would degrade. So each sub-researcher runs in its own private context, and only its short summary leaves it, controlled by two settings on the delegation tool: `outputs_to_string` (what the orchestrator sees) and `outputs_to_state` (what's appended to `notes` for the writer). The bulky raw research never propagates to the orchestrator or the writer." ] }, { "cell_type": "markdown", "id": "8d3eb1d3", "metadata": {}, "source": [ "### Customizing the agent\n", "\n", "Each phase takes its own `ChatGenerator`, so you can mix models by cost and capability, or swap in a different provider entirely:\n", "\n", "- `scope_llm`, `orchestrator_llm`, `writer_llm`: default to `OpenAIResponsesChatGenerator(\"gpt-5.4\")` (the heavier reasoning steps).\n", "- `researcher_llm`, `summarizer_llm`: default to `OpenAIResponsesChatGenerator(\"gpt-5.4-mini\")` (run many times, so a cheaper model keeps cost down).\n", "\n", "And the breadth/depth of the investigation is fully tunable: `max_subtopics`, `max_concurrent_researchers`, `max_orchestrator_steps`, `max_researcher_steps`, `max_search_results`, and `max_content_length`.\n", "\n", "For example, to make the whole run cheaper you might point every phase at a smaller model:\n", "\n", "```python\n", "from haystack.components.generators.chat import OpenAIChatGenerator\n", "from haystack_integrations.agent_pack import create_deep_research_agent\n", "\n", "cheap_agent = create_deep_research_agent(\n", " scope_llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " orchestrator_llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " researcher_llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " summarizer_llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " writer_llm=OpenAIChatGenerator(model=\"gpt-4o-mini\"),\n", " max_subtopics=3,\n", ")\n", "```" ] }, { "cell_type": "markdown", "id": "aa45f1d4", "metadata": {}, "source": [ "## Agent Pack as a blueprint\n", "\n", "Beyond running and customizing, the third way to use the pack is to **copy it**. Both agents are built entirely from public Haystack primitives, so their source doubles as a reference architecture:\n", "\n", "- The Deep Research Agent shows how to nest agents (a sub-agent exposed to the orchestrator as a `ComponentTool`) and how `outputs_to_string` and `outputs_to_state` control what a sub-agent returns to the caller versus what it saves for later, the core of its context isolation.\n", "- The Advanced RAG Agent shows how to build tools over a document store, embed a grammar in a tool's parameter description, accumulate results in `State`, and use an `after_run` hook as a safety net.\n", "\n", "Browse the source in [`haystack-core-integrations/integrations/agent_pack`](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack) and adapt whichever parts fit your use case." ] }, { "cell_type": "markdown", "id": "94f5479d", "metadata": {}, "source": [ "## What's next\n", "\n", "🎉 Congratulations! You've run and customized both agents in Agent Pack, seen how they work under the hood, and how they can serve as blueprints for your own architectures.\n", "\n", "**Realistically, what to expect from Agent Pack going forward:** it's an early, experimental project, so treat these agents as a fast-moving starting point rather than a stable API. Expect the `create_*` signatures and internals to change as the patterns settle (the customization interface in particular is still being refined), so pin a version if you build on it. The pack lives outside `haystack-ai` on purpose so it can move quickly, and it doubles as a place to prototype capabilities that may later graduate into Haystack itself. More agents are likely to land here over time.\n", "\n", "The most useful thing you can do is give feedback: if you hit a bug, or have an idea for a complex agent that belongs in the pack, [open an issue](https://github.com/deepset-ai/haystack-core-integrations/issues). Keep an eye on the [documentation](https://docs.haystack.deepset.ai/docs/agent-pack) and the [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/agent_pack) for updates.\n", "\n", "To go deeper on building agents from scratch, check out these tutorials:\n", "\n", "- [Build a Tool-Calling Agent](https://haystack.deepset.ai/tutorials/43_building_a_tool_calling_agent)\n", "- [Creating a Multi-Agent System with Haystack](https://haystack.deepset.ai/tutorials/45_creating_a_multi_agent_system)\n", "- [Human-in-the-Loop with Haystack Agents](https://haystack.deepset.ai/tutorials/47_human_in_the_loop_agent)\n", "\n", "To stay up to date on the latest Haystack developments, you can [sign up for our newsletter](https://landing.deepset.ai/haystack-community-updates) or [join Haystack discord community](https://discord.gg/Dr63fr9NDS)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }