Haystack
Get Enterprise Support
Get Started
🆕 Haystack 3.1 is here! Context compaction, token counters, and AgentTool for multi-agent systems

Integration: Arize AI

Trace and Monitor your Haystack pipelines with Arize AI

Authors
Arize AI
Arize-ai
GitHub Repo PyPI Package

Table of Contents

Overview

Arize AI helps teams observe, evaluate, and improve LLM applications and AI agents. Use Arize AX for the full-featured platform built for production teams, AI-native companies, and enterprises, available as managed cloud or enterprise self-hosted deployment. Use Arize Phoenix when you need an open-source workflow for local development, experimentation, or self-hosting.

Arize’s agent evaluation guide and LLM evaluation guide show how traces can support debugging, evaluation, and continuous quality monitoring.

Installation

pip install openinference-instrumentation-haystack haystack-ai arize-otel opentelemetry-sdk opentelemetry-exporter-otlp

Usage

To trace any Haystack pipeline with Arize, simply initialize OpenTelemetry and the HaystackInstrumentor. Haystack pipelines that run within the same environment send traces to Arize.

from openinference.instrumentation.haystack import HaystackInstrumentor
# Import open-telemetry dependencies
from arize_otel import register_otel, Endpoints

# Setup OTEL via our convenience function
register_otel(
    endpoints = Endpoints.ARIZE,
    space_id = "<your-space-id>", # from the space settings page
    api_key = "<your-api-key>", # from the space settings page
    model_id = "<your-haystack-app-name>", # name this to whatever you would like
)

Now, you can run a Haystack pipeline within the same environment, resulting in the following trace:

To run the example below, export your OpenAI Key to the OPENAI_API_KEY environment variable.

Arize Demo 

from haystack import Document, Pipeline
from haystack.components.builders.prompt_builder import PromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore

document_store = InMemoryDocumentStore()
document_store.write_documents([
    Document(content="My name is Jean and I live in Paris."),
    Document(content="My name is Mark and I live in Berlin."),
    Document(content="My name is Giorgio and I live in Rome.")
])

prompt_template = """
Given these documents, answer the question.
Documents:
{% for doc in documents %}
    {{ doc.content }}
{% endfor %}
Question: {{question}}
Answer:
"""

retriever = InMemoryBM25Retriever(document_store=document_store)
prompt_builder = PromptBuilder(template=prompt_template)
llm = OpenAIChatGenerator()

rag_pipeline = Pipeline()
rag_pipeline.add_component("retriever", retriever)
rag_pipeline.add_component("prompt_builder", prompt_builder)
rag_pipeline.add_component("llm", llm)
rag_pipeline.connect("retriever", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder", "llm")

question = "Who lives in Paris?"
results = rag_pipeline.run(
    {
        "retriever": {"query": question},
        "prompt_builder": {"question": question},
    }
)