Integration: SerperDev
Uses Serper.dev API for web searches
Table of Contents
Overview
The SerperDevWebSearch component allows you to perform web searches using the
Serper.dev API. It retrieves relevant snippets and URLs that can be used directly in your Haystack applications, such as Retrieval-Augmented Generation (RAG) pipelines or with Haystack Agents.
This component is part of the core haystack-ai package, meaning you do not need to install an external integration package to use it.
When you pass a query to SerperDevWebSearch, it returns a list of URLs and text snippets that are most relevant to your search.
Installation
Since the Serper.dev web search component is built into the core Haystack framework, you just need to install haystack-ai:
pip install haystack-ai
You will also need to get an API key from Serper.dev and set it as an environment variable:
export SERPERDEV_API_KEY="your-api-key"
Usage
Components
This integration provides the following component:
-
SerperDevWebSearch: A component that queries the Serper.dev API to find web pages relevant to a given query.
Standalone Usage
Here is how you can use SerperDevWebSearch directly:
from haystack.components.websearch import SerperDevWebSearch
from haystack.utils import Secret
import os
# Ensure the environment variable is set
os.environ["SERPERDEV_API_KEY"] = "your-api-key"
# Initialize the component (it automatically uses the SERPERDEV_API_KEY environment variable)
web_search = SerperDevWebSearch(api_key=Secret.from_env_var("SERPERDEV_API_KEY"))
# Run a search query
results = web_search.run(query="What is Haystack AI?")
# Access the search results
documents = results["documents"]
links = results["links"]
for doc in documents:
print(f"Title: {doc.meta.get('title')}")
print(f"Snippet: {doc.content}")
print(f"URL: {doc.meta.get('link')}")
print("-" * 50)
Usage within a Pipeline
You can integrate SerperDevWebSearch directly into a Generative QA pipeline to answer questions using the live web context:
import os
from haystack import Pipeline
from haystack.components.websearch import SerperDevWebSearch
from haystack.components.builders import ChatPromptBuilder
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage
os.environ["SERPERDEV_API_KEY"] = "your-serperdev-api-key"
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
template = [
ChatMessage.from_user("""
Given the following web search results, answer the user's question.
Search Results:
{% for document in documents %}
{{ document.content }}
{% endfor %}
Question: {{ query }}
Answer:
""")
]
pipe = Pipeline()
pipe.add_component("websearch", SerperDevWebSearch())
pipe.add_component("prompt_builder", ChatPromptBuilder(template=template))
pipe.add_component("llm", OpenAIChatGenerator())
pipe.connect("websearch.documents", "prompt_builder.documents")
pipe.connect("prompt_builder.prompt", "llm.messages")
query = "What are the latest features in Haystack 2.0?"
response = pipe.run({
"websearch": {"query": query},
"prompt_builder": {"template_variables": {"query": query}}
})
print(response["llm"]["replies"][0].content)
License
haystack-ai is distributed under the terms of the
Apache-2.0 license.