Prompts

Learn how to configure and manage prompts in your R2R deployment

Prompt Management in R2R

R2R provides a flexible system for managing prompts, allowing you to create, update, retrieve, and delete prompts dynamically. This system is crucial for customizing the behavior of language models and ensuring consistent interactions across your application.

Default Prompts

R2R comes with a set of default prompts that are loaded from YAML files located in the py/core/providers/prompts/defaults directory. These default prompts provide a starting point for various tasks within the R2R system.

For example, the default RAG (Retrieval-Augmented Generation) prompt is defined as follows:

default_rag:
  template: >
    ## Task:

    Answer the query given immediately below given the context which follows later. Use line item references to like [1], [2], ... refer to specifically numbered items in the provided context. Pay close attention to the title of each given source to ensure it is consistent with the query.


    ### Query:

    {query}


    ### Context:

    {context}


    ### Query:

    {query}


    REMINDER - Use line item references to like [1], [2], ... refer to specifically numbered items in the provided context.

    ## Response:
  input_types:
    query: str
    context: str

Default Prompt Usage

Here’s a table showing the purpose of some key default prompts:

Certainly! I’ll create an expanded table that explains all the prompts you’ve listed, with links to their respective GitHub files. Here’s the updated table:

Prompt File	Purpose
default_rag.yaml	Default prompt for Retrieval-Augmented Generation (RAG) tasks. It instructs the model to answer queries based on provided context, using line item references.
few_shot_ner_kg_extraction.yaml	Used for few-shot Named Entity Recognition (NER) and Knowledge Graph (KG) extraction. It provides examples to guide the model in identifying entities and relationships.
few_shot_ner_kg_extraction_with_spec.yaml	Similar to the above, but includes a specific schema or specification for the extraction process.
graphrag_community_reports.yaml	Used in GraphRAG to generate reports about communities or clusters in the knowledge graph.
graphrag_map_system_prompt.yaml	System prompt for the “map” phase in GraphRAG, used to process individual nodes or edges.
graphrag_reduce_system_prompt.yaml	System prompt for the “reduce” phase in GraphRAG, used to combine or summarize information from multiple sources.
graphrag_triples_extraction_few_shot.yaml	Few-shot prompt for extracting subject-predicate-object triplets in GraphRAG, with examples.
graphrag_triples_extraction_few_shot.yaml	Zero-shot prompt for extracting subject-predicate-object triplets in GraphRAG, without examples.
hyde.yaml	Related to Hypothetical Document Embeddings (HyDE) for improving retrieval performance.
kg_search.yaml	Used for searching the knowledge graph, possibly to formulate queries or interpret results.
kg_search_with_spec.yaml	Similar to kg_search.yaml, but with a specific schema or specification for the search process.
rag_agent.yaml	Defines the behavior and instructions for the RAG agent, which coordinates the retrieval and generation process.
rag_context.yaml	Used to process or format the context retrieved for RAG tasks.
rag_fusion.yaml	Used in RAG fusion techniques, possibly for combining information from multiple retrieved passages.
system.yaml	Contains general system-level prompts or instructions for the R2R system.
zero_shot_ner_kg_extraction.yaml	Used for zero-shot Named Entity Recognition and Knowledge Graph extraction, without providing examples.
zero_shot_ner_kg_extraction_with_spec.yaml	Similar to the above, but includes a specific schema or specification for the extraction process.

You can find the full list of default prompts and their contents in the defaults directory.

Prompt Provider

R2R uses a R2RPromptProvider class to manage prompts. This provider handles the storage, retrieval, and manipulation of prompts, leveraging both a PostgreSQL database and YAML files for flexibility and persistence.

Key features of the R2RPromptProvider:

1. Database Storage: Prompts are stored in a PostgreSQL table, allowing for efficient querying and updates.
2. YAML File Support: Prompts can be loaded from YAML files, providing an easy way to version control and distribute default prompts.
3. In-Memory Cache: Prompts are kept in memory for fast access during runtime.

Prompt Structure

Each prompt in R2R consists of:

• Name: A unique identifier for the prompt.
• Template: The actual text of the prompt, which may include placeholders for dynamic content.
• Input Types: A dictionary specifying the expected types for any dynamic inputs to the prompt.

Managing Prompts

R2R provides several endpoints and SDK methods for managing prompts:

Adding a Prompt

To add a new prompt:

from r2r import R2RClient

client = R2RClient()

response = client.add_prompt(
    name="my_new_prompt",
    template="Hello, {name}! Welcome to {service}.",
    input_types={"name": "str", "service": "str"}
)

Updating a Prompt

To update an existing prompt:

response = client.update_prompt(
    name="my_existing_prompt",
    template="Updated template: {variable}",
    input_types={"variable": "str"}
)

Retrieving a Prompt

To get a specific prompt:

response = client.get_prompt(
    prompt_name="my_prompt",
    inputs={"variable": "example"},
    prompt_override="Optional override text"
)

Listing All Prompts

To retrieve all prompts:

response = client.get_all_prompts()

Deleting a Prompt

To delete a prompt:

response = client.delete_prompt("prompt_to_delete")

Security Considerations

Access to prompt management functions is restricted to superusers to prevent unauthorized modifications to system prompts. Ensure that only trusted administrators have superuser access to your R2R deployment.

Best Practices

1. Version Control: Store your prompts in version-controlled YAML files for easy tracking of changes and rollbacks.
2. Consistent Naming: Use a consistent naming convention for your prompts to make them easy to identify and manage.
3. Input Validation: Always specify input types for your prompts to ensure that they receive the correct data types.
4. Regular Audits: Periodically review and update your prompts to ensure they remain relevant and effective.
5. Testing: Test prompts thoroughly before deploying them to production, especially if they involve complex logic or multiple input variables.

Advanced Usage

Dynamic Prompt Loading

R2R’s prompt system allows for dynamic loading of prompts from both the database and YAML files. This enables you to:

1. Deploy default prompts with your application.
2. Override or extend these prompts at runtime.
3. Easily update prompts without redeploying your entire application.

Prompt Templating

The prompt template system in R2R supports complex string formatting. You can include conditional logic, loops, and other Python expressions within your prompts using a templating engine.

Example of a more complex prompt template:

complex_template = """
Given the following information:
{% for item in data %}
- {{ item.name }}: {{ item.value }}
{% endfor %}

Please provide a summary that {% if include_analysis %}includes an analysis of the data{% else %}lists the key points{% endif %}.
"""

client.add_prompt(
    name="complex_summary",
    template=complex_template,
    input_types={"data": "list", "include_analysis": "bool"}
)

This flexibility allows you to create highly dynamic and context-aware prompts that can adapt to various scenarios in your application.

Conclusion

R2R’s prompt management system provides a powerful and flexible way to control the behavior of language models in your application. By leveraging this system effectively, you can create more dynamic, context-aware, and maintainable AI-powered features.

For more detailed information on other aspects of R2R configuration, please refer to the following pages:

• LLM Configuration
• RAG Configuration
• Postgres Configuration
• Ingestion Configuration