Maintained by deepset

Integration: Amazon Bedrock

Use Models from AI21 Labs, Anthropic, Cohere, Meta, and Amazon via Amazon Bedrock with Haystack


Table of Contents


Amazon Bedrock is a fully managed service that makes high-performing foundation models from leading AI startups and Amazon available for your use through a unified API. You can choose from various foundation models to find the one best suited for your use case. More information can be found on the Amazon Bedrock documentation page.


Install the Amazon Bedrock integration:

pip install amazon-bedrock-haystack


Once installed, you will have access to AmazonBedrockGenerator and AmazonBedrockChatGenerator components that support foundation models on Amazon Bedrock.


To use this integration for text generation, initialize an AmazonBedrockGenerator with the model name, the AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION) should be set as environment variables or passed as Secret arguments. Note, make sure the region you set supports Amazon Bedrock.

Currently, the following models are supported:

  • AI21 Labs’ Jurassic-2
  • Amazon Titan language models
  • Anthropic’s Claude
  • Cohere’s Command
  • Meta’s Llama 2
from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockGenerator

generator = AmazonBedrockGenerator(model="anthropic.claude-v2")
result ="Who is the best American actor?")
for reply in result["replies"]:


'There is no definitive "best" American actor, as acting skill and talent a# re subjective. However, some of the most acclaimed and influential American act# ors include Tom Hanks, Daniel Day-Lewis, Denzel Washington, Meryl Streep, Rober# t De Niro, Al Pacino, Marlon Brando, Jack Nicholson, Leonardo DiCaprio and John# ny Depp. Choosing a single "best" actor comes down to personal preference.'


To use this integration for chat models, initialize an AmazonBedrockChatGenerator with the model name and aws credentials:

Currently, the following models are supported:

  • Anthropic’s Claude
  • Meta’s Llama 2
from haystack_integrations.components.generators.amazon_bedrock import AmazonBedrockChatGenerator
from haystack.dataclasses import ChatMessage
generator = AmazonBedrockChatGenerator(model="meta.llama2-70b-chat-v1")
messages = [ChatMessage.from_system("You are a helpful assistant that answers question in Spanish only"), 
            ChatMessage.from_user("What's Natural Language Processing? Be brief.")]
response =


{'replies': [ChatMessage(content='  Procesamiento del Lenguaje Natural (PLN) es una rama de la inteligencia artificial que se enfoca en el desarrollo de algoritmos y modelos computacionales para analizar, comprender y generar texto y lenguaje humano.', role=<ChatRole.ASSISTANT: 'assistant'>, name=None, meta={'prompt_token_count': 46, 'generation_token_count': 60, 'stop_reason': 'stop'})]}