Haystack
🆕 Meet the provider-agnostic tool-calling Agent and SuperComponent. Go to 2.12 release notes for all details 🌟

Haystack 2.12.0

Check on Github

⭐️ Highlights

Agent Component with State Management

The Agent component enables tool-calling functionality with provider-agnostic chat model support and can be used as a standalone component or within a pipeline. With SERPERDEV_API_KEY and OPENAI_API_KEY defined, a Web Search Agent is as simple as:

from haystack.components.agents import Agent 
from haystack.components.generators.chat import OpenAIChatGenerator 
from haystack.components.websearch import SerperDevWebSearch 
from haystack.dataclasses import ChatMessage 
from haystack.tools.component_tool import ComponentTool 

web_tool = ComponentTool(     
    component=SerperDevWebSearch(), 
) 

agent = Agent(     
    chat_generator=OpenAIChatGenerator(),
    tools=[web_tool],
) 

result = agent.run(
    messages=[ChatMessage.from_user("Find information about Haystack by deepset")]
)

The Agent supports streaming responses, customizable exit conditions, and a flexible state management system that enables tools to share and modify data during execution:

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[web_tool, weather_tool],
    exit_conditions=["text", "weather_tool"],
    state_schema = {...},
    streaming_callback=streaming_callback,
)

SuperComponent for Reusable Pipelines

SuperComponent allows you to wrap complex pipelines into reusable components. This makes it easy to reuse them across your applications. Just initialize a SuperComponent with a pipeline:

from haystack import Pipeline, SuperComponent

with open("pipeline.yaml", "r") as file:
  pipeline = Pipeline.load(file)

super_component = SuperComponent(pipeline)

That’s not all! To show the benefits, there are three ready-made SuperComponents in haystack-experimental. For example, there is a MultiFileConverter that wraps a pipeline with converters for CSV, DOCX, HTML, JSON, MD, PPTX, PDF, TXT, and XSLX. After installing the integration dependencies pip install pypdf markdown-it-py mdit_plain trafilatura python-pptx python-docx jq openpyxl tabulate, you can run with any of the supported file types as input:

from haystack_experimental.super_components.converters import MultiFileConverter

converter = MultiFileConverter()
converter.run(sources=["test.txt", "test.pdf"], meta={})

Here’s an example of creating a custom SuperComponent from any Haystack pipeline:

from haystack import Pipeline, SuperComponent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.builders import ChatPromptBuilder
from haystack.components.retrievers import InMemoryBM25Retriever
from haystack.dataclasses.chat_message import ChatMessage
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.dataclasses import Document

document_store = InMemoryDocumentStore()
documents = [
    Document(content="Paris is the capital of France."),
    Document(content="London is the capital of England."),
]
document_store.write_documents(documents)

prompt_template = [
    ChatMessage.from_user(
    '''
    According to the following documents:
    {% for document in documents %}
    {{document.content}}
    {% endfor %}
    Answer the given question: {{query}}
    Answer:
    '''
    )
]
prompt_builder = ChatPromptBuilder(template=prompt_template, required_variables="*")

pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", OpenAIChatGenerator())
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder.prompt", "llm.messages")

# Create a super component with simplified input/output mapping
wrapper = SuperComponent(
    pipeline=pipeline,
    input_mapping={
        "query": ["retriever.query", "prompt_builder.query"],
    },
    output_mapping={"llm.replies": "replies"}
)

# Run the pipeline with simplified interface
result = wrapper.run(query="What is the capital of France?")
print(result)
# {'replies': [ChatMessage(_role=<ChatRole.ASSISTANT: 'assistant'>,
#  _content=[TextContent(text='The capital of France is Paris.')],...)

⬆️ Upgrade Notes

  • Updated ChatMessage serialization and deserialization. ChatMessage.to_dict() now returns a dictionary with the keys: role, content, meta, and name. ChatMessage.from_dict() supports this format and maintains compatibility with older formats.

    If your application consumes the result of ChatMessage.to_dict(), update your code to handle the new format. No changes are needed if you’re using ChatPromptBuilder in a Pipeline.

  • LLMEvaluator, ContextRelevanceEvaluator, and FaithfulnessEvaluator now internally use a ChatGenerator instance instead of a Generator instance. The public attribute generator has been replaced with _chat_generator.

  • to_pandas, comparative_individual_scores_report and score_report were removed from EvaluationRunResult, please use detailed_report, comparative_detailed_report and aggregated_report instead.

🚀 New Features

  • Treat bare types (e.g., List, Dict) as generic types with Any arguments during type compatibility checks.
  • Add compatibility for Callable types.
  • Adds outputs_to_string to Tool and ComponentTool to allow users to customize how the output of a Tool should be converted into a string so that it can be provided back to the ChatGenerator in a ChatMessage. If outputs_to_string is not provided, a default converter is used within ToolInvoker. The default handler uses the current default behavior.
  • Added a new parameter split_mode to the CSVDocumentSplitter component to control the splitting mode. The new parameter can be set to row-wise to split the CSV file by rows. The default value is threshold, which is the previous behavior.
  • We added a new retrieval technique, AutoMergingRetriever which together with the HierarchicalDocumentSplitter implement a auto-merging retrieval technique.
  • Add run_async method to HuggingFaceLocalChatGenerator. This method internally uses ThreadPoolExecutor to return coroutines that can be awaited.
  • Introduced asynchronous functionality and HTTP/2 support in the LinkContentFetcher component, thus improving content fetching in several aspects.
  • The DOCXToDocument component now has the option to include extracted hyperlink addresses in the output Documents. It accepts a link_format parameter that can be set to “markdown” or “plain”. By default, no hyperlink addresses are extracted as before.
  • Added a new parameter azure_ad_token_provider to all Azure OpenAI components: AzureOpenAIGenerator, AzureOpenAIChatGenerator, AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder. This parameter optionally accepts a callable that returns a bearer token, enabling authentication via Azure AD.
    • Introduced the utility function default_azure_token_provider in haystack/utils/azure.py. This function provides a default token provider that is serializable by Haystack. Users can now pass default_azure_token_provider as the azure_ad_token_provider or implement a custom token provider.
  • Users can now work with date and time in the ChatPromptBuilder. In the same way as the PromptBuilder, the ChatPromptBuilder now supports arrow to work with datetime.
  • Introduce new State dataclass with a customizable schema for managing Agent state. Enhance error logging of Tool and extend the ToolInvoker component to work with newly introduced State.
  • The RecursiveDocumentSplitter now supports splitting by number of tokens. Setting “split_unit” to “token” will use a hard-coded tiktoken tokenizer (o200k_base) and requires having tiktoken installed.

⚡️ Enhancement Notes

  • LLMEvaluator, ContextRelevanceEvaluator, and FaithfulnessEvaluator now accept a chat_generator initialization parameter, consisting of ChatGenerator instance pre-configured to return a JSON object. Previously, these components only supported OpenAI and LLMs with OpenAI-compatible APIs. Regardless of whether the evaluator components are initialized with api, api_key, and api_params or the new chat_generator parameter, the serialization format will now only include chat_generator in preparation for the future removal of api, api_key, and api_params.
  • Improved error handling for component run failures by raising a runtime error that includes the component’s name and type.
  • When using Haystack’s Agent, the messages are stored and accumulated in State. This means:
    • State is required to have a “messages” type and handler defined in its schema. If not provided, a default type and handler are provided. Users can now customize how messages are accumulated by providing a custom handler for messages through the State schema.
  • Added PDFMinerToDocument functionality to detect and report undecoded CID characters in PDF text extraction, helping users identify potential text extraction quality issues when processing PDFs with non-standard fonts.
  • The Agent component allows defining multiple exit conditions instead of a single condition. The init parameter has been renamed from exit_condition to exit_conditions to reflect that.
  • Introduce a ChatGenerator Protocol to qualify ChatGenerator components from a static type-checking perspective. It defines the minimal interface that Chat Generators must implement. This will especially help to standardize the integration of Chat Generators into other more complex components.
  • In Agent, we check all messages from the LLM when doing an exit condition check. For example, it’s possible the LLM returns multiple messages, such as multiple tool calls, or includes messages with reasoning. Now we check all messages before assessing if we should exit the loop.
  • The Agent component checks whether the ChatGenerator it is initialized with supports tools. If it doesn’t, the Agent raises a TypeError.
  • Updated SentenceTransformersDiversityRanker to use the token parameter internally instead of the deprecated use_auth_token. The public API of this component already utilizes token.
  • Simplified the serialization code for better readability and maintainability.
    • Updated deserialization to allow users to omit the typing. prefix for standard typing library types (e.g., List[str] instead of typing.List[str]).
  • Refactored the processing of streaming chunks from OpenAI to simplify logic.
    • Added tests to ensure expected behavior when handling streaming chunks when using include_usage=True.
  • Updates the doc strings of the BranchJoiner to more understandable and better highlight where it’s useful.
  • Consolidate the use of select_streaming_callback utility in OpenAI and Azure ChatGenerators, which checks the compatibility of streaming_callback with the async or non-async run method.
  • Added a warning to ChatPromptBuilder and PromptBuilder when prompt variables are present and required_variables is unset to help users avoid unexpected execution in multi-branch pipelines. The warning recommends users to set required_variables.

⚠️ Deprecation Notes

  • The api, api_key, and api_params parameters for LLMEvaluator, ContextRelevanceEvaluator, and FaithfulnessEvaluator are now deprecated and will be removed in Haystack 2.13.0. By default, these components will continue to use OpenAI in JSON mode. To configure a specific LLM, use the chat_generator parameter.
  • The generator_api and generator_api_params initialization parameters of LLMMetadataExtractor and the LLMProvider enum are deprecated and will be removed in Haystack 2.13.0. Use chat_generator instead to configure the underlying LLM. For example, change generator_api=LLMProvider.OPENAI to chat_generator=OpenAIChatGenerator().

🐛 Bug Fixes

  • Add dataframe to legacy fields for the Document dataclass. This fixes a bug where Document.from_dict() in haystack-ai>=2.11.0 could not properly deserialize a Document dictionary obtained with document.to_dict(flatten=False) in haystack-ai<=2.10.0.
  • In DALLEImageGenerator, ensure that the max_retries initialization parameter is correctly set when it is equal to 0.
  • Fixed an index error in the logging module when arbitrary strings are logged.
  • Ensure that the max_retries initialization parameter is correctly set when equal 0 in AzureOpenAIGenerator, AzureOpenAIChatGenerator, AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder.
  • Improved serialization and deserialization in haystack/utils/type_serialization.py to handle Optional types correctly.
  • Replace lazy imports with eager imports in haystack/__init__.py to avoid potential static type checking issues and simplify maintenance.
  • Fix an issue that prevented Jinja2-based ComponentTools from being passed into pipelines at runtime.
  • Improved type hinting for the component.output_types decorator. The type hinting for the decorator was originally introduced to avoid overshadowing the type hinting of the run method and allow proper static type checking. This update extends support to asynchronous run_async methods.
  • Fixed issue with MistralChatGenerator not returning a finish_reason when using streaming. Fixed by adjusting how we look for the finish_reason when processing streaming chunks. Now, the last non-None finish_reason is used to handle differences between OpenAI and Mistral.