Knowledge Graph
Learn how to configure and use the knowledge graph provider in R2R
Introduction
R2R’s KGProvider
handles the creation, management, and querying of knowledge graphs in your applications. This guide offers an in-depth look at the system’s architecture, configuration options, and best practices for implementation.
For a practical, step-by-step guide on implementing knowledge graphs in R2R, including code examples and common use cases, see our GraphRAG Cookbook.
Configuration
Knowledge Graph Configuration
These are located in the r2r.toml
file, under the [kg]
section.
[kg]
provider = "postgres"
batch_size = 256
[kg.kg_creation_settings]
kg_triples_extraction_prompt = "graphrag_triples_extraction_few_shot"
entity_types = ["Person", "Organization", "Location"] # if empty, all entities are extracted
relation_types = ["works at", "founded by", "invested in"] # if empty, all relations are extracted
max_knowledge_triples = 100
fragment_merge_count = 4 # number of fragments to merge into a single extraction
generation_config = { model = "openai/gpt-4o-mini" } # and other params, model used for triplet extraction
[kg.kg_enrichment_settings]
max_description_input_length = 65536 # increase if you want more comprehensive descriptions
max_summary_input_length = 65536 # increase if you want more comprehensive summaries
generation_config = { model = "openai/gpt-4o-mini" } # and other params, model used for node description and graph clustering
leiden_params = {}
Environment variables take precedence over the config settings in case of conflicts. The R2R Docker includes configuration options that facilitate integration with a combined Postgres+pgvector database setup.
Implementation Guide
File Ingestion and Graph Construction
from r2r import R2RClient
client = R2RClient("http://localhost:7272")
result = client.ingest_files(["path/to/your/file.txt"])
# following will create a graph on all ingested files
document_ids = [] # add document ids that you want to create a graph on
creation_result = client.create_graph(document_ids)
print(f"Creation Result: {creation_result}")
# wait for the creation to complete
enrichment_result = client.enrich_graph() # enrichment will run on all nodes in the graph
print(f"Enrichment Result: {enrichment_result}")
# wait for the enrichment to complete
Graph-based Search
There are two types of graph-based search: local
and global
.
local
search is faster and more accurate, but it is not as comprehensive asglobal
search.global
search is slower and more comprehensive, but it will give you the most relevant results. Note that global search may perform a large number of LLM calls.
search_result = client.search(
query="Find founders who worked at Google",
kg_search_settings={"use_kg_search":True, "kg_search_type": "local"}
)
print(f"Search Result: {search_result}")
Retrieval-Augmented Generation
rag_result = client.rag(
query="Summarize the achievements of founders who worked at Google",
kg_search_settings={"use_kg_search":True, "kg_search_type": "local"}
)
print(f"RAG Result: {rag_result}")
Best Practices
- Optimize Chunk Size: Adjust the
chunk_size
based on your data and model capabilities. - Use Domain-Specific Entity Types and Relations: Customize these for more accurate graph construction.
- Balance Batch Size: Adjust
batch_size
for optimal performance and resource usage. - Implement Caching: Cache frequently accessed graph data for improved performance.
- Regular Graph Maintenance: Periodically clean and optimize your knowledge graph.
Advanced Topics
Custom Knowledge Graph Providers
Extend the KGProvider
class to implement custom knowledge graph providers:
from r2r.base import KGProvider, KGConfig
class CustomKGProvider(KGProvider):
def __init__(self, config: KGConfig):
super().__init__(config)
# Custom initialization...
def ingest_files(self, file_paths: List[str]):
# Custom implementation...
def search(self, query: str, use_kg_search: bool = True):
# Custom implementation...
# Implement other required methods...
Integrating External Graph Databases
To integrate with external graph databases:
- Implement a custom
KGProvider
. - Handle data synchronization between R2R and the external database.
- Implement custom querying methods to leverage the external database’s features.
Scaling Knowledge Graphs
For large-scale applications:
- Implement graph partitioning for distributed storage and processing.
- Use graph-specific indexing techniques for faster querying.
- Consider using a graph computing framework for complex analytics.
Troubleshooting
Common issues and solutions:
- Ingestion Errors: Check file formats and encoding.
- Query Performance: Optimize graph structure and use appropriate indexes.
- Memory Issues: Adjust batch sizes and implement pagination for large graphs.
Conclusion
R2R’s Knowledge Graph system provides a powerful foundation for building applications that require structured data representation and complex querying capabilities. By understanding its components, following best practices, and leveraging its flexibility, you can create sophisticated information retrieval and analysis systems tailored to your specific needs.
For further customization and advanced use cases, refer to the R2R API Documentation and the GraphRAG Cookbook.