Haystack Hub REST API

We also provide a Swagger UI. Please feel free to check it out.

API Reference

The Haystack Hub API is organized around REST. Our API has predictable resource-oriented URLs, accepts form-encoded request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs.

Authentication

Create a new API token

Parameters
Endpoint
post /api/v1/auth/token
Attributes
username string
Email address of the user

password string
Login password

expire_in
(Optional) Expiry time of token in seconds

Request Body Example
{
  "username": "tim.smith@abc.com",
  "password": "User Password",
  "expire_in": 3600
}

Blacklist an existing API token

Parameters
Endpoint
post /api/v1/auth/blacklist
Attributes
access_token string
Bearer token to blacklist

example

Request Body Example

Errors

Haystack Hub Rest API uses conventional HTTP response codes to indicate the success or failure of an API request. In general: Codes in the 2xx range indicate success. Codes in the 4xx range indicate an error that failed given the information provided. Codes in the 5xx range indicate an error with Haystack Hub's servers.

HTTP Status Code Summary
200 - OKEverything worked as expected.
400 - Bad RequestThe request was unacceptable, often due to missing a required parameter.
401 - UnauthorizedNo valid API key provided.
402 - Request FailedThe parameters were valid but the request failed.
403 - ForbiddenThe API key doesn't have permissions to perform the request.
404 - Not FoundThe requested resource doesn't exist.
409 - ConflictThe request conflicts with another request (perhaps due to using the same idempotent key).
429 - Too Many RequestsToo many requests hit the API too quickly. We recommend an exponential backoff of your requests.
500, 502, 503, 504 - Server ErrorsSomething went wrong on Haystack Hub's end.

User

Invite new users to the organization

Parameters
Endpoint
post /api/v1/users
Attributes
email string
The email of the user who was invited

workspaces array

is_super_admin boolean
The user is a super_admin within the organisation (true/false)

Request Body Example
{
  "email": "tim.smith@abc.com",
  "workspaces": [
    {
      "name": "default",
      "role": "developer"
    }
  ],
  "is_super_admin": true
}

Get all users in the organization

Parameters
NameExample
limit (integer)10
page_number (integer)1
Endpoint
get /api/v1/users
Returns

Returns a list of all users within the company.

Response
{
  "data": [
    {
      "user_id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
      "email": "tim.smith@abc.com",
      "first_name": "Tim",
      "last_name": "Smith",
      "is_super_admin": true,
      "workspaces": [
        {
          "name": "default",
          "role": "developer"
        }
      ]
    }
  ]
}

Get details of an user

Parameters
NameExample
user_id * (string)141dbaad-3156-4885-a34e-888667f9f6d9
Endpoint
get /api/v1/users/{user_id}
Returns

Returns details of the user which has the given id

Response
{
  "user_id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
  "email": "tim.smith@abc.com",
  "first_name": "Tim",
  "last_name": "Smith",
  "is_super_admin": true,
  "workspaces": [
    {
      "name": "default",
      "role": "developer"
    }
  ]
}

Edit details of an user

Parameters
NameExample
user_id * (string)141dbaad-3156-4885-a34e-888667f9f6d9
Endpoint
put /api/v1/users/{user_id}
Returns

Return the updated user details of the given user

Response
{
  "user_id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
  "email": "tim.smith@abc.com",
  "first_name": "Tim",
  "last_name": "Smith",
  "is_super_admin": true,
  "workspaces": [
    {
      "name": "default",
      "role": "developer"
    }
  ]
}

Get details of current user

Parameters
Endpoint
get /api/v1/me
Returns

Returns the details of the logged in user

Response
{
  "user_id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
  "email": "tim.smith@abc.com",
  "first_name": "Tim",
  "last_name": "Smith",
  "is_super_admin": true,
  "workspaces": [
    {
      "name": "default",
      "role": "developer"
    }
  ]
}

Sign up as a new organization

Parameters
Endpoint
post /api/v1/signup
Attributes
email string
The user's email address

first_name string
First name

last_name string
Last name

organization_name string
The organization the user is working for

password string
Password defined by the user

Request Body Example
{
  "email": "tim.smith@abc.com",
  "first_name": "Tim",
  "last_name": "Smith",
  "organization_name": "Your GmbH",
  "password": "s€cur€_p9SS!"
}

Change password

Parameters
Endpoint
post /api/v1/change_password
Attributes
old_password string
Old password

new_password string
New password

Request Body Example
{
  "old_password": "old_pa$$",
  "new_password": "new_pa$$"
}

Forgot the login password

Parameters
Endpoint
post /api/v1/forgot_password
Attributes
email string
The user's email address

Request Body Example
{
  "email": "tim.smith@abc.com"
}

Reset the login password

Parameters
Endpoint
post /api/v1/reset_password
Attributes
new_password string
New password

reset_id string
Unique password reset id

reset_key string
Password reset key

Request Body Example
{
  "new_password": "s€cur€_p9SS!2.0",
  "reset_id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
  "reset_key": "kjh4jadKdenF23"
}

Reset the login password

Parameters
Endpoint
post /api/v1/verify_email
Attributes
verification_key string
Uniquev key to verify the request

Request Body Example
{
  "verification_key": "lKJfdkkj839jh5"
}

Resend the verification email for your account

Parameters
Endpoint
post /api/v1/auth/resend_verify_email
Attributes
Request Body Example
{}

Configuration

Get retriever of workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/retrievers
Returns

Returns the available retrievers of the workspace

Response
{
  "data": [
    {
      "retriever": {
        "type": "DensePassageRetriever",
        "description": "Dense Passage Retrieval",
        "is_recommended": true,
        "models": [
          {
            "name": "deepset/roberta-base-squad2",
            "description": "RoBERTa model trained on SQuAD",
            "is_recommended": true
          }
        ]
      }
    }
  ]
}

Get readers of workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/readers
Returns

Returns the available readers of the workspace

Response
{
  "data": [
    {
      "type": "FARMReader",
      "description": "FARM Reader",
      "is_recommended": true,
      "models": [
        {
          "name": "deepset/roberta-base-squad2",
          "description": "RoBERTa model trained on SQuAD",
          "is_recommended": true
        }
      ]
    }
  ]
}

Get the configuration of a Workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/config
Returns

Returns the workspace configuration which is defined by the workspace_name

Response
{
  "reader": {
    "type": "FARMReader",
    "model": {
      "name": "deepset/roberta-base-squad2",
      "amp": false
    },
    "top_k": 10,
    "no_ans_boost": -10,
    "context_window_size": 500,
    "doc_stride": 128,
    "max_seq_len": 256
  },
  "retriever": {
    "model": {
      "name": "deepset/roberta-base-squad2"
    },
    "top_k": 10,
    "type": "DensePassageRetrieval"
  },
  "file": {
    "remove_numeric_tables": false,
    "remove_whitespace": false,
    "remove_empty_lines": false,
    "remove_header_footer": false
  }
}

Change the configuration of a Workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
patch /api/v1/workspaces/{workspace_name}/config
Attributes
reader object

retriever object

file object

Request Body Example
{
  "reader": {
    "type": "FARMReader",
    "model": {
      "name": "deepset/roberta-base-squad2",
      "amp": false
    },
    "top_k": 10,
    "no_ans_boost": -10,
    "context_window_size": 500,
    "doc_stride": 128,
    "max_seq_len": 256
  },
  "retriever": {
    "model": {
      "name": "deepset/roberta-base-squad2"
    },
    "top_k": 10,
    "type": "DensePassageRetrieval"
  },
  "file": {
    "remove_numeric_tables": false,
    "remove_whitespace": false,
    "remove_empty_lines": false,
    "remove_header_footer": false
  }
}

Workspace

List all Workspaces

Parameters
Endpoint
get /api/v1/workspaces
Returns

Returns all workspaces of the organization

Response
{
  "data": [
    {
      "name": "dafault",
      "role": "developer"
    }
  ]
}

Get a workspace by name

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}
Returns

Returns the workspace defined by the given name

Response
{
  "name": "dafault",
  "role": "developer"
}

Dashboard

Get history of search queries

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/search_history
Returns

Returns the search history of the workspace

Response
{
  "data": [
    {
      "request": {
        "questions": [
          "Who is Arya's sister?"
        ],
        "filters": null,
        "top_k_reader": 5,
        "top_k_retriever": 5
      }
    },
    {
      "response": {
        "question": "Who is Arya's sister?",
        "answer": "Sansa",
        "context": "\n====Season 1====\nArya accompanies her father Ned and her sister Sansa to King's Landing. Before their departure, Arya's half-brother Jon Snow gifts Arya a sword which she dubs \"Needle\". On the Kingsroad, Arya is sparring with a butcher's boy, Mycah, when Sansa's betrothed Prince Joffrey Baratheon attacks Mycah, prompting Arya's direwolf Nymeria to bite Joffrey. Arya shoos Nymeria away so she is not killed, but is furious when Sansa later refuses to support her version of events. Mycah is later ",
        "file": {
          "id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
          "name": "berlin.pdf"
        },
        "offset_end": 71,
        "offset_end_in_doc": 71,
        "offset_start": 65,
        "offset_start_in_doc": 65,
        "probability": 0.8372742768459336,
        "score": 13.104684829711914
      }
    },
    {
      "user": {
        "id": "7da504ef-d825-4e89-b479-cac538c9df7b",
        "name": "Tim Smith"
      },
      "duration": 2342,
      "time": "2018-12-19"
    }
  ],
  "has_more": true
}

Get day-wise count of search queries for the last month

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/search_count
Returns

Returns the search queries per day of the last month

Response
{
  "data": [
    {
      "date": "2018-12-19",
      "count": 73
    }
  ]
}

Get workspace query statistics

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/stats
Returns

Returns workspace query statistics

Response
{
  "file_count": 245,
  "document_count": 2310,
  "average_response_time": 1.12
}

Files

Upload files to workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
post /api/v1/workspaces/{workspace_name}/files
Attributes
file string
A file in pdf or txt format. Optionally, if working with custom text extraction from an external source like HTML or database, a plain-text string can also be passed here in the file parameter. In such cases, the `file_name` parameter can be used to set a custom name for the uploaded string.

file_name
Override the name of uploaded file or set the file name in case a string is uploaded.

meta
Serialized JSON string of key/value(both of type string) pairs that can later be used for filtering in search.

split_by
'word', 'sentence', or 'passage'

split_length
Maximum number of the above split unit (e.g. word) that are allowed in one document

split_respect_sentence_boundary
only valid when split_by is set to 'word'

remove_empty_lines boolean
Remove multiple blank lines when processing files.

remove_header_footer boolean
Use heuristics to remove repeating header/footer string in files.

remove_numeric_tables boolean
Use heuristics to remove numeric tabular data from the files.

remove_whitespace boolean
Strip any additional whitespaces in the files.

valid_languages array
Validate language of the files to be one from the configured language for the Workspace.

Request Body Example
{
  "file": "Berlin.pdf",
  "file_name": "Berlin.pdf",
  "meta": "{}",
  "split_by": "word",
  "split_length": 500,
  "split_respect_sentence_boundary": false,
  "remove_empty_lines": false,
  "remove_header_footer": false,
  "remove_numeric_tables": false,
  "remove_whitespace": false
}

List files in Workspace

Parameters
NameExample
workspace_name * (string)default
limit (integer)10
page_number (integer)1
Endpoint
get /api/v1/workspaces/{workspace_name}/files
Returns

Returns all files within the workspace

Response
{
  "data": [
    {
      "id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
      "url": "http://link-to-download-file",
      "name": "Berlin.pdf",
      "size": 16,
      "characters": 2050,
      "languages": [
        "en"
      ]
    }
  ]
}

Preview file before upload

Parameters
NameExample
workspace_name * (string)default
Endpoint
post /api/v1/workspaces/{workspace_name}/file-preview
Attributes
file string
A file in pdf or txt format. Optionally, if working with custom text extraction from an external source like HTML or database, a plain-text string can also be passed here in the file parameter. In such cases, the `file_name` parameter can be used to set a custom name for the uploaded string.

file_name
Override the name of uploaded file or set the file name in case a string is uploaded.

meta
Serialized JSON string of key/value(both of type string) pairs that can later be used for filtering in search.

split_by
'word', 'sentence', or 'passage'

split_length
Maximum number of the above split unit (e.g. word) that are allowed in one document

split_respect_sentence_boundary
only valid when split_by is set to 'word'

remove_empty_lines boolean
Remove multiple blank lines when processing files.

remove_header_footer boolean
Use heuristics to remove repeating header/footer string in files.

remove_numeric_tables boolean
Use heuristics to remove numeric tabular data from the files.

remove_whitespace boolean
Strip any additional whitespaces in the files.

valid_languages array
Validate language of the files to be one from the configured language for the Workspace.

Request Body Example
{
  "file": "Berlin.pdf",
  "file_name": "Berlin.pdf",
  "meta": "{}",
  "split_by": "word",
  "split_length": 500,
  "split_respect_sentence_boundary": false,
  "remove_empty_lines": false,
  "remove_header_footer": false,
  "remove_numeric_tables": false,
  "remove_whitespace": false
}

Download file from Workspace

Parameters
NameExample
workspace_name * (string)default
file_id * (string)d594d351-1661-4cf2-b542-4e0e4f1ffe4b
Endpoint
get /api/v1/workspaces/{workspace_name}/files/{file_id}
Returns

Downloads the file workspace_name

Response

Delete file from Workspace

Parameters
NameExample
workspace_name * (string)default
file_id * (string)d594d351-1661-4cf2-b542-4e0e4f1ffe4b
Endpoint
delete /api/v1/workspaces/{workspace_name}/files/{file_id}
Returns

File was deleted successfully

Get documents for a file

Parameters
NameExample
workspace_name * (string)default
file_id * (string)d594d351-1661-4cf2-b542-4e0e4f1ffe4b
Endpoint
get /api/v1/workspaces/{workspace_name}/files/{file_id}/documents
Returns

Returns all documents for the file with the File ID file_id

Response
{
  "data": [
    {
      "text": "Sample document text"
    }
  ]
}

Get metadata for a file

Parameters
NameExample
workspace_name * (string)default
file_id * (string)d594d351-1661-4cf2-b542-4e0e4f1ffe4b
Endpoint
get /api/v1/workspaces/{workspace_name}/files/{file_id}/meta
Returns

Returns all meta data for the file with the File ID file_id

Response
{
  "author": "Tim Smith",
  "year": "2020"
}

Update metadata for a file

Parameters
NameExample
workspace_name * (string)default
file_id * (string)d594d351-1661-4cf2-b542-4e0e4f1ffe4b
Endpoint
put /api/v1/workspaces/{workspace_name}/files/{file_id}/meta
Returns

Returns the updated meta data for the file with the ID file_id

Response
{
  "author": "Tim Smith",
  "year": "2020"
}

Get meta data fields for a Workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/metadata_fields
Returns

Returns all meta data field of the workspace

Response
{
  "data": [
    {
      "name": "Author",
      "type": "string"
    }
  ]
}

Update the meta data fields for a Workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
put /api/v1/workspaces/{workspace_name}/metadata_fields
Returns

Returns the updated meta fields of the workspace

Response
{
  "data": [
    {
      "name": "Author",
      "type": "string"
    }
  ]
}

List files with indexing errors

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/languages
Returns

Returns all available languages for the workspace

Response
{
  "properties": {
    "languages": {
      "type": "array",
      "items": {
        "$ref": "#/components/schemas/Language"
      }
    }
  }
}

List files with indexing errors

Parameters
NameExample
workspace_name * (string)default
Endpoint
get /api/v1/workspaces/{workspace_name}/invalid_files
Returns

Returns all files with an indexing error

Response
{
  "data": [
    {
      "id": "d594d351-1661-4cf2-b542-4e0e4f1ffe4b",
      "url": "http://link-to-download-file",
      "name": "Berlin.pdf",
      "size": 16,
      "characters": 2050,
      "languages": [
        "en"
      ],
      "reason": "Invalid Language"
    }
  ]
}

Search answers in Workspace

Parameters
NameExample
workspace_name * (string)default
Endpoint
post /api/v1/workspaces/{workspace_name}/search
Attributes
questions array
List of questions

Request Body Example
{
  "questions": [
    "Who is Arya's sister?"
  ]
}

Organization

Get details of the user's organization

Parameters
Endpoint
get /api/v1/organization
Returns

Returns the details of the user's organization

Response
{
  "name": "deepset"
}

Request a new subscription plan (starter, basic, premium or enterprise)

Parameters
Endpoint
post /api/v1/auth/request_subscription
Attributes
pricing_plan string
subscription level (starter, basic, premium or enterprise)

Request Body Example
{
  "pricing_plan": "starter"
}
© 2020 - 2021 deepset. All rights reserved.Imprint