> ## Documentation Index
> Fetch the complete documentation index at: https://r2r-patch-fix-ingestion.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Embedding

## Embedding Provider

By default, R2R uses the LiteLLM framework to communicate with various cloud embedding providers. To customize the embedding settings:

```toml r2r.toml
[embedding]
provider = "litellm"
base_model = "openai/text-embedding-3-small"
base_dimension = 512
batch_size = 128
add_title_as_prefix = false
rerank_model = "None"
concurrent_request_limit = 256
```

Let's break down the embedding configuration options:

* `provider`: Choose from `ollama`, `litellm` and `openai`. R2R defaults to using the LiteLLM framework for maximum embedding provider flexibility.
* `base_model`: Specifies the embedding model to use. Format is typically "provider/model-name" (e.g., `"openai/text-embedding-3-small"`).
* `base_dimension`: Sets the dimension of the embedding vectors. Should match the output dimension of the chosen model.
* `batch_size`: Determines the number of texts to embed in a single API call. Larger values can improve throughput but may increase latency.
* `add_title_as_prefix`: When true, prepends the document title to the text before embedding, providing additional context.
* `rerank_model`: Specifies a model for reranking results. Set to "None" to disable reranking (note: not supported by LiteLLMEmbeddingProvider).
* `concurrent_request_limit`: Sets the maximum number of concurrent embedding requests to manage load and avoid rate limiting.

<Note> Embedding providers for an R2R system cannot be configured at runtime and are instead configured server side. </Note>

### Supported  LiteLLM Providers

Support for any of the embedding providers listed below is provided through LiteLLM.

<Tabs>
  <Tab title="OpenAI">
    Example configuration:

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "openai/text-embedding-3-small"
    base_dimension = 512
    ```

    ```bash
    export OPENAI_API_KEY=your_openai_key
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * text-embedding-3-small
    * text-embedding-3-large
    * text-embedding-ada-002

    For detailed usage instructions, refer to the [LiteLLM OpenAI Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#openai-embedding-models).
  </Tab>

  <Tab title="Azure">
    Example configuration:

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "azure/<your deployment name>"
    base_dimension = XXX
    ```

    ```bash
    export AZURE_API_KEY=your_azure_api_key
    export AZURE_API_BASE=your_azure_api_base
    export AZURE_API_VERSION=your_azure_api_version
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * text-embedding-ada-002

    For detailed usage instructions, refer to the [LiteLLM Azure Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#azure-openai-embedding-models).
  </Tab>

  <Tab title="Anthropic">
    Anthropic does not currently offer embedding models. Consider using OpenAI or another provider for embeddings.
  </Tab>

  <Tab title="Cohere">
    Example configuration:

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "cohere/embed-english-v3.0"
    base_dimension = 1_024
    ```

    ```bash
    export COHERE_API_KEY=your_cohere_api_key
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * embed-english-v3.0
    * embed-english-light-v3.0
    * embed-multilingual-v3.0
    * embed-multilingual-light-v3.0
    * embed-english-v2.0
    * embed-english-light-v2.0
    * embed-multilingual-v2.0

    For detailed usage instructions, refer to the [LiteLLM Cohere Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#cohere-embedding-models).
  </Tab>

  <Tab title="Ollama">
    When running with Ollama, additional changes are recommended for the to the `r2r.toml` file. In addition to using the `ollama` provider directly, we recommend restricting the `concurrent_request_limit` in order to avoid exceeding the throughput of your Ollama server.

    ```toml example r2r.toml
    [embedding]
    provider = "ollama"
    base_model = "ollama/mxbai-embed-large"
    base_dimension = 1_024
    batch_size = 32
    add_title_as_prefix = true
    ```

    ```bash
    # Ensure your Ollama server is running
    # Default Ollama server address: http://localhost:11434
    # <-- OR -->
    # Use `r2r --config-name=local_llm serve --docker`
    # which bundles ollama with R2R in Docker by default!

    r2r serve --config-path=r2r.toml
    ```

    Then, deploy your R2R server with `r2r serve --config-path=r2r.toml `.
  </Tab>

  <Tab title="HuggingFace">
    Example configuration:

    ```toml example r2r.toml
    [embedding]
    provider = "litellm"
    base_model = "huggingface/microsoft/codebert-base"
    base_dimension = 768
    ```

    ```python
    export HUGGINGFACE_API_KEY=your_huggingface_api_key

    r2r serve --config-path=r2r.toml
    ```

    LiteLLM supports all Feature-Extraction Embedding models on HuggingFace.

    For detailed usage instructions, refer to the [LiteLLM HuggingFace Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#huggingface-embedding-models).
  </Tab>

  <Tab title="Bedrock">
    Example configuration:

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "bedrock/amazon.titan-embed-text-v1"
    base_dimension = 1_024
    ```

    ```bash
    export AWS_ACCESS_KEY_ID=your_access_key
    export AWS_SECRET_ACCESS_KEY=your_secret_key
    export AWS_REGION_NAME=your_region_name
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * amazon.titan-embed-text-v1
    * cohere.embed-english-v3
    * cohere.embed-multilingual-v3

    For detailed usage instructions, refer to the [LiteLLM Bedrock Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#bedrock-embedding).
  </Tab>

  <Tab title="Vertex AI">
    Example configuration:

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "vertex_ai/textembedding-gecko"
    base_dimension = 768
    ```

    ```bash
    export GOOGLE_APPLICATION_CREDENTIALS=path/to/your/credentials.json
    export VERTEX_PROJECT=your_project_id
    export VERTEX_LOCATION=your_project_location
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * textembedding-gecko
    * textembedding-gecko-multilingual
    * textembedding-gecko\@001
    * textembedding-gecko\@003
    * text-embedding-preview-0409
    * text-multilingual-embedding-preview-0409

    For detailed usage instructions, refer to the [LiteLLM Vertex AI Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#vertex-ai-embedding-models).
  </Tab>

  <Tab title="Voyage AI">
    Example Configuration

    ```toml example r2r.toml
    provider = "litellm"
    base_model = "voyage/voyage-01"
    base_dimension = 1_024
    ```

    ```bash
    export VOYAGE_API_KEY=your_voyage_api_key
    # .. set other environment variables

    r2r serve --config-path=r2r.toml
    ```

    Supported models include:

    * voyage-01
    * voyage-lite-01
    * voyage-lite-01-instruct

    For detailed usage instructions, refer to the [LiteLLM Voyage AI Embedding documentation](https://docs.litellm.ai/docs/embedding/supported_embedding#voyage-ai-embedding-models).
  </Tab>
</Tabs>